Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI and Nvidia announce partnership to deploy 10GW of Nvidia systems

openai.com

Post reply on HN
Print view View original

371–380 of 668 posts

Re: OpenAI and Nvidia announce partnership to deploy 10GW of Nvidia systems

#371

Post by ComputerGuru »

Framing it in gigawatts is very interesting given the controversy about skyrocketing electric prices for residential and small business users as a result of datacenters over the past three years, primarily driven by AI growth. If, as another commenter notes, this 10GW is how much Chicago and NYC use combined, then we need to have a serious discussion about where this power is going to come from given the dismal status of the USA's power grid and related infrastructure and the already exploding costs that have been shifted to residential users in order to guarantee electric supply to these biggest datacenters (so they can keep paying peanuts for electricity and avoid shouldering any of the infrastructural burden to maintain or improve the underlying grid/plants required to meet their massive power needs).

I'm not even anti-datacenter (wouldn't be here if I were), I just think there needs to be serious rebalancing of these costs because this increase in US residential electric prices in just five years (from 13¢ to 19¢, a ridiculous 46% increase) is neither fair nor sustainable.

So where is this 10GW electric supply going to come from and who is going to pay for it?

Source: https://fred.stlouisfed.org/series/APU000072610

EDIT:

To everyone arguing this is how DCs are normally sized: yes, but normally it's not the company providing the compute for the DC owner that is giving these numbers. nVidia doesn't sell empty datacenters with power distribution networks, cooling, and little else; nVidia sells the GPUs that will stock that DC. This isn't a typical PR netnewswire bulletin "OpenAI announces new 10GW datacenter", this is "nvidia is providing xx compute for OpenAI". Anyway, all this is a segue from the question of power supply, consumption, grid expansion/stability, and who is paying for all that.

Re: OpenAI and Nvidia announce partnership to deploy 10GW of Nvidia systems

#372

Post by javiramos »

JCM9 wrote: post #108

This is throwing more cards on the house of cards. Nvidia is “investing” in OpenAI so OpenAI can buy GPUs from NVidia. Textbook “round tripping.” I generally like what’s been happening with AI but man this is gonna crash hard when reality sets in. We’re reaching the scary stage of a bubble where folks are forced to throw more and more cash on the fire to keep it going with no clear path to ever get that cash back. If…

"...man this is gonna crash hard when reality sets in."

Given the amount of money invested and the expectations, the crash will be of cataclysmic proportions

Re: OpenAI and Nvidia announce partnership to deploy 10GW of Nvidia systems

#373

Post by NooneAtAll3 »

dcchambers wrote: post #330
gloxkiqcza wrote: post #282

I’m really curious how this affects the consumer GPU market over the next few years. Sure, there has been a GPU shortage for a few years now but if this continues, there should be an absolute surplus of obsolete-gen enterprise GPUs flooding the market, right? Any ideas what limitations and benefits these cards might have for an enthusiast?

I feel like we're lucky Nvidia even sells consumer GPUs any more. At this point it's just a distraction to them and takes away resources they could be devoting to higher value hardware. And the data center-class hardware doesn't do well in a home environment. It's not good for gaming. It runs hot and uses a ton of energy. Not to mention, silicon that is running hot 24/7 for years probably isn't the best thing to own…

probably explains why we have "just rent H100 in the cloud, duh" influencers under every hardware building post on hn now

Re: OpenAI and Nvidia announce partnership to deploy 10GW of Nvidia systems

#374

Post by lawlessone »

sindriava wrote: post #291
lawlessone wrote: post #224

Earlier quoted context omitted.

any examples?

Current systems are already tremendously useful in the medical field. And I'm not talking about your doctor asking ChatGPT random shit, I'm saying radiology results processing, patient monitoring, monitoring of medication studies... The list goes on. Not to mention many of the research advances done using automated systems already, for example for weather forecasting.

Ok, but i am asking for uses for LLMs specifically.

Of course i agree ML has already helped in many other areas and has a bright future. But the thing everyone is talking about here are LLM's

Re: OpenAI and Nvidia announce partnership to deploy 10GW of Nvidia systems

#375

Post by 0xTJ »

Stating compute scale in terms of power consumption is such a backwards metric to me, assuming that you're trying to portray is as something positive.

It's like selling steel by the average fractional number of mining deaths that went into producing it. Sure, at a given moment there will be some ratio between average deaths and steel, but that's a number that you want to be as low as possible.

Re: OpenAI and Nvidia announce partnership to deploy 10GW of Nvidia systems

#376

Post by crowcroft »

littlestymaar wrote: post #364
paganel wrote: post #339

Earlier quoted context omitted.

> debt is highly explosive (see GE during the GFC) whereas equity is not. Not as explosive as debt but I'd venture to say that nowadays equity is a lot more "inflamable" compared to 2008-2010, as in a lot more debt-like (which I think partly explains the current equity bubble in the US). As in, there are lots and lots of investment funds/pension funds/other such like financial entities which are very heavily tied to…

> As in, there are lots and lots of investment funds/pension funds/other such like financial entities which are very heavily tied to the "performance" of equity, and I'm talking about trillions (at this point) of dollars, and if that equity were to get a, let's say, 20 or 30% hair-cut in a matter of two-three months (at most), then we'll for sure be back in October 2008 mode. Just curious, can you detail how it would…

Anytime there's a massive draw down equities an asset-liability mismatch shows up (margin calls) because someone was borrowing money to spend in the short term against the value of assets that have now disappeared.

It might not be the catastrophic cascading failure of the GFC, but someone somewhere in the pile will get exposed.

Re: OpenAI and Nvidia announce partnership to deploy 10GW of Nvidia systems

#377

Post by crowcroft »

paganel wrote: post #339
crowcroft wrote: post #306

Earlier quoted context omitted.

It seems similar to how GE under Jack Welch would use their rock solid financials to take on low cost debt that they could lend out to suppliers who needed finance to purchase their products. The biggest difference here though is that most of these moves seem to to involve direct investment and the movement of equity, not debt . I think this is an important distinction, because if things take a downturn debt is highl…

> debt is highly explosive (see GE during the GFC) whereas equity is not. Not as explosive as debt but I'd venture to say that nowadays equity is a lot more "inflamable" compared to 2008-2010, as in a lot more debt-like (which I think partly explains the current equity bubble in the US). As in, there are lots and lots of investment funds/pension funds/other such like financial entities which are very heavily tied to…

Totally agree, it might not blow up in Nvidia's face, but there's a margin call sitting around somewhere in the pile.

Re: OpenAI and Nvidia announce partnership to deploy 10GW of Nvidia systems

#378

Post by CodingJeebus »

pmdr wrote: post #194

If I had shovels to sell, I'd definitely announce a strategic partnership to have a huge quarry dug by hand. Seriously, is there anyone in the media keeping unbiased tabs on how much we're spending on summarizing emails and making creatives starve a little more?

Ed Zitron is an AI skeptic from the market perspective, highly recommend his stuff. It’s definitely not comforting to read, but he’s doing the math behind these headlines and it’s not adding up at all[0]

0: https://www.wheresyoured.at/

Re: OpenAI and Nvidia announce partnership to deploy 10GW of Nvidia systems

#379

Post by mgh95 »

crowcroft wrote: post #363
mgh95 wrote: post #347

Earlier quoted context omitted.

"Marginally ROI positive" works in a ZIRP environment. These are huge capital investments; they need to at least clear treasury return hurdles and importantly provide attractive returns . I am fundamentally skeptical of "scaling inference". Margins are not defensible in the market segment OpenAI is in.

For some of these tech companies their valuations let them go to the market with their equity in way that is basically a ZIRP environment. In a way you could say this is a competitive advantage someone like Nvidia has at the moment and so they are trying to push that. I'm also pretty skeptical, and could imagine this whole thing blowing up, but it's not like this a big grift that's going to end up like the GFC either…

I think it's possible we are in datacenter GPU overcapacity already, and NVIDIA is burning its stock to avoid the music stopping.

It's already happening in China that datacenters are at GPU overcapacity. I wouldn't be surprised if it occurs here.

Re: OpenAI and Nvidia announce partnership to deploy 10GW of Nvidia systems

#380

Post by lacy_tinpot »

binarymax wrote: post #337
lacy_tinpot wrote: post #328

Earlier quoted context omitted.

This is more like reinvesting into the business as it's growing. It's a positive sum loop. Nvidia makes money by selling to OpenAI. OpenAI makes money by selling a service to users that uses Nvidia. So Nvidia invests in the build out and expansion of the infrastructure that will use Nvidia. This is a classic positive sum loop. It's not that different than a company reinvesting revenue in growing the company.

But is OpenAI recouping this? I remember seeing reports a year ago that it was in the realm of $700M/mo in inference costs for them - are they earning that now? Of course the strategy of taking a loss and reinvesting - but I don't see how OpenAI is making enough money to pay for all this, now or in the future.

It hasn't monetized any of its services. There are currently no ads. And it's not selling user data, well not yet.

That's literally hundreds of billions worth of revenue.

Just look at the options OpenAI has to generate revenue beyond subscriptions.

Post reply on HN