So Nvidia is giving OpenAI money so OpenAI can buy more Nvidia GPUs?
OpenAI and Nvidia announce partnership to deploy 10GW of Nvidia systems
491–500 of 668 posts
Re: OpenAI and Nvidia announce partnership to deploy 10GW of Nvidia systems
#492Re: OpenAI and Nvidia announce partnership to deploy 10GW of Nvidia systems
#493Framing it in gigawatts is very interesting given the controversy about skyrocketing electric prices for residential and small business users as a result of datacenters over the past three years, primarily driven by AI growth. If, as another commenter notes, this 10GW is how much Chicago and NYC use combined , then we need to have a serious discussion about where this power is going to come from given the dismal stat…
Utilities always need to justify rate increases with the regulator.
The bulk of cost increases come from the transition to renewable energy. You can check your local utility and see.
It’s very easy to make a huge customer like a data center directly pay the cost needed to serve them from the grid.
Generation of electricity is more complicated, the data centers pulling cheap power from Colombia river hydro are starting to compete with residential users.
Generation is a tiny fraction of electricity charges though.
Re: OpenAI and Nvidia announce partnership to deploy 10GW of Nvidia systems
#494Earlier quoted context omitted.
Before reading your comment I did some napkin math using 600W per GPU: 10,000,000,000 / 600 = 16,666,666.66... With varying consumption/TDP, could be significantly more, could be significantly less, but at least it gives a starting figure. This doesn't account for overhead like energy losses, burst/nominal/sustained, system overhead, and heat removal.
B200 is 1kW+ TDP ;)
And consists of 8 GPUs
Re: OpenAI and Nvidia announce partnership to deploy 10GW of Nvidia systems
#495Earlier quoted context omitted.
I work in the datacenter space. The power consumption of a data center is the "canonical" way to describe their size. Almost every component in a datacenter is upgradeable—in fact, the compute itself only has a lifespan of ~5 years—but the power requirements are basically locked-in. A 200MW data center will always be a 200MW data center, even though the flops it computes will increase. The fact that we use this unit…
It is the opposite of refining energy. Electrical energy is steak, what leaves the datacenter is heat, the lowest form of energy that we might still have a use for in that concentration (but most likely we are just dumping it in the atmosphere). Refining is taking a lower quality energy source and turning it into a higher quality one. What you could argue is that it adds value to bits. But the bits themselves, their…
I think you're pushing the metaphor a bit far, but the parallel was to something like ore.
A power plant "mines" electron, which the data center then refines into words. or whatever. The point is that energy is the raw material that flows into data centers.
Re: OpenAI and Nvidia announce partnership to deploy 10GW of Nvidia systems
#496To the people who are calling this evidence of a bubble: There is no credible indication that AI in general is a bubble, even if not all investments will make sense in retrospect. Quite the opposite, the progress in the field over the last few years is staggering. AI systems are becoming superhuman at more and more tasks. It's only a question of time till AI will outperform us at everything.
> There is no credible indication that AI in general is a bubble, even if not all investments will make sense in retrospect. If you add up all of the contracts that OpenAI is signing, it's buying something like $1 trillion/year worth of compute. To merely break even, it would have to make more money than literally every other company on the planet, fairly close to twice the current highest revenue company (Walmart, a…
They are aiming at being the first to develop an AGI and eventually superintelligence. Something that can replace human workers. Walmart is small fish in comparison. OpenAI is currently in the lead, so their chances are decent.
Re: OpenAI and Nvidia announce partnership to deploy 10GW of Nvidia systems
#497For someone who doesn't know what a gigawat worth of Nvidia systems is, how many high-end H100 or whatever does this get you? My estimates along with some poor-grade GPT research leads me to think it could be nearly 10 million? That does seem insane.
Before reading your comment I did some napkin math using 600W per GPU: 10,000,000,000 / 600 = 16,666,666.66... With varying consumption/TDP, could be significantly more, could be significantly less, but at least it gives a starting figure. This doesn't account for overhead like energy losses, burst/nominal/sustained, system overhead, and heat removal.
It's easy to see what it could be by looking at Green500.
Re: OpenAI and Nvidia announce partnership to deploy 10GW of Nvidia systems
#498To put this into perspective this datacenter would have the land area of Monaco (740 acres) given assumptions of a 80kW/rack per case.
To put Monaco in perspective, the US could fit 4.8M Monacos
To put Rhode Island in perspective, it would hold 1,340 Monacos.
Re: OpenAI and Nvidia announce partnership to deploy 10GW of Nvidia systems
#499Earlier quoted context omitted.
All of those expenses could be trimmed in a scenario where OpenAI or other big labs pivot to focus primarily on profitability via selling inference.
Currently, selling LLM inference is a red queen race: the moment you release a model, others begin distilling and attempting to sell your model cheaper, avoiding the expensive capitalized costs associated with R&D. This can occur because the LLM market is fundamentally -- at best -- minimally differentiated; consumers are willing to switch between vendors ("big labs", as you call them, but they aren't really research…
I don't disagree but you're moving the goalposts. I never said that they could achieve the profits of a typical tech business, just that they could be profitable. Also, the whole distilling problem doesn't happen if the model is proprietary.
Re: OpenAI and Nvidia announce partnership to deploy 10GW of Nvidia systems
#500Earlier quoted context omitted.
Where is this water meme coming from? Surely the water is just pumped around, not actually used up?
Evaporative cooling effectively "uses up" the water. It's possible to run chillers instead, but that consumes more electricity, and some power plants also use evaporative cooling.
Some water usage has highly questionable counting methodologies.
Like using if a datacenter is using hydroelectric power you count the evaporation from the dam reservoir as "used water".
I'm not an expert but imo correct accounting should really only consider direct consumption. It's very silly when we play games like having petro states have very high carbon footprints even if they don't actually burn the fuel.