Dumb question, but when the Nebius capacity dashboard says they have around 3 non-preemptible B200s available, does that mean _total_, or is it just how many I myself might be able to rent on demand?
One aspect of the profitability might be the utilization and the pricing a few years down the line for slightly older hardware. Already now it seems like the increased processing you get from newer devices versus the cost difference makes something like an H100 or even A100 significantly less desirable than newer more powerful ones. As an individual, I am happy to be able to get an H200 on demand, but the B200 or B300 can do so much more work with optimized software and models for only modestly more cost that if those become available then from a business perspective you really have to prefer that if you can keep it occupied.
Then with Vera Rubin being like 3 times more effective or whatever, that adds a new layer of gradual obsolescence. So the question is can they keep the pricing up on the older ones a few years down the line enough to fill out the end of those expected payback periods.
The real boogeyman for a neocloud that has heavily invested in expensive Nvidia hardware might be a variation of that beyond Nvidia with startups that have even more dramatic efficiency increases pushing the leading edge even further. For example, if companies like Mythic AI and d-Matrix could somehow rapidly rapidly scale, that would push prices down for all of Nvidia hardware that is significantly less efficient.
I guess so far it doesn't look like any startups with really big efficiency breakthroughs are even close to being able to scale like Nvidia though, especially with the manufacturing and power crunch. But I suspect some of that is because of favoritism and strong arming protecting investments rather than a free and fair ecosystem.