ClusterMAX debrief: >100 days later

Source: Quok.it

Written by Ahmed Abdellah

It’s been about four months since SemiAnalysis shook the neocloud industry by publishing ClusterMAX. ClusterMAX was the first third-party rating system that systematically evaluated the quality of service provided by different neoclouds. The outcome? Developers realized they were getting screwed. Billion-dollar training efforts suddenly became vulnerable. And the whole world was asking— what’s next?

Since then, a lot has changed. Many players in this space formed alliances, and others pivoted altogether. All while geopolitics injected even greater chaos.

In this article, we’ll explore the major developments that have occurred since the original publication, as well as evaluation criteria we plan to expand upon at Quok.it.

Neocloud Consolidation & Growth

Since ClusterMAX’s publication in March, roughly 10% of the reported clouds have either been acquired, pivoted, or sunsetted.

Source: Quok.it

At first glance, it may appear that the industry is starting to consolidate. In reality, quite the opposite is true. We see new neoclouds, marketplaces, and even decentralized physical infrastructure networks (DePINs) popping up daily (too many to touch on here). The market for AI infrastructure has become a full-blown gladiator arena. Neoclouds face a steady stream of new challengers, but the competition is fierce, and many newcomers are unable to survive for long.

While they’re busy fighting each other, a shifting power dynamic between chipmakers and data centers could send it all crashing down.

NVIDIA tightens its grip

While alternative chipmakers have been making headlines (FuriosaAI & LG deal, Google TPU DoD grant winner, largest AMD cloud deployed by Tensorwave), NVIDIA took neoclouds by storm by releasing their own cloud, DGX Cloud.

Source: Quok.it

Their message to neocloud providers was simple: either use us, or compete with us. Participation in NVIDIA’s preferred partner program now mandates that some capacity must be reserved for DGX Cloud. NVIDIA’s actions are aggressive: after five years of helping cloud providers procure hardware, they’re now threatening the existence of these providers. We’re already getting a glimpse of the reactions to this.

We’ve seen some neoclouds, such as Yotta, embrace the partnership adjustment. On the other hand, we’ve also seen many players in the space begin to diversify away from NVIDIA to protect their core interests. Players like DigitalOcean have shifted towards hosting MI300x and MI325x GPUs, and heterogeneous clouds like Cirrascale have grown to offer AMD, Cerebras, and Qualcomm chips. Google’s also showing no signs of slowing down in their race to break free from NVIDIA with the release of the Ironwood TPU in April.

AMD had a massive opportunity here to align itself with soured neoclouds— but instead, they decided to release their own developer cloud in partnerships with DigitalOcean. In my opinion, this was a terrible decision. This decision not only alienated the other (very few) AMD clouds, but it was also a wasted opportunity to poach NVIDIA Neoclouds, which were threatened by the DGX cloud. AMD needs all the allies it can get, especially since the AMD team allegedly doesn’t have access to its own R&D clusters… lol

Source: South Park

Quok.it filling in the Gap

There are many additional evaluation metrics of cloud providers that haven’t yet been addressed. Over the coming months, we’ll focus on evaluating developer experience, API quality, reliability, and transparency across cloud providers and marketplaces.

Developer experience

We’ll be testing how easy it is to create a new account, spin up an instance, and determine who gives out free credits (and who misrepresents them @GPU.net). Many of these will be picky, but the goal here is to help cloud providers reduce as much friction as possible to get developers using their instances.

Source: GPUnet official Telegram

API Quality

We’ve been documenting our own API experiences across different cloud providers and some are quite atrocious. We will be publicly naming and shaming different providers in the coming months. If you’d like our feedback before hand, please reach out.

Transparency

There’s a lot of confusion around who actually owns their GPUs. Many current marketplaces masquerade as a neocloud and offer very little value beyond the BareMetal instances. We believe these marketplaces should be isolated from those that actually provide value to developers (e.g., serverless, good DevEX, fast inference, etc.). This space needs a lot more transparency regarding the ownership of hardware. It’s confusing when you have data centers like Applied Digital reviewing other data centers like Flexential. Are they co-locating or sharing capacity? It would be nice to know.

DePIN

Then there’s DePIN…There’s a lot to unpack in this space, so we will be conducting a deep dive of these players and separating the few gems from the sheer garbage. Just to give you a taste of what we’re dealing with here, Spheron network has spent the past FIVE YEARS boasting about building the “world’s largest community-powered data center” just to finally reveal they pull GPUs from a whopping… TWO providers (Sesterce and Voltage Park— both centralized data centers). Please change your bio.

Source: Spheron Network

It’s still a very confusing space to navigate, but winners will start to emerge over the coming years. Our thesis is that it won’t be by deploying more capacity, raising more debt, or launching fancy videos. It’ll be through transparency, reliability, and excellent developer experience.

We’re excited to see the expanded list in the second iteration of ClusterMAX. We believe that some major players, such as VoltagePark (Neocloud) and Hyperbolic (Marketplace), were missed in the first edition, which we believe would have performed quite well. We’d love to see them included in the next iteration.

If you’re a neocloud and want to share your thoughts on DGX Cloud to me anonymously, please contact us or email wonzo@quok.it.