91–100 of 124 posts
Datacenters didn't need water cooling before the AI explosion. (Air cooling was still possible) At first, DW's estimate was one drop of potable water was consumed for each query (normal queries, not more expensive ones) The Google, I don't know who allowed the sincerity, God bless him, released a first hand analysis of their water consumption, and it is higher that the one drop estimate: 5 drops https://services.goog…
It's been a while, but I don't recall any of the dotcom startups making deals with nuclear energy companies to buy out entire nuclear power stations: https://www.npr.org/2024/09/20/nx-s1-5120581/three-mile-isla... And that's just an example, there are many power-related deals of similar magnitude. The companies building out capacity certainly believe that AI is going to use as much power as we are told. We are told t…
> The companies building out capacity certainly believe that AI is going to use as much power as we are told. The same could be said of dark fiber laid during the dot com boom, or unused railroads, etc. Spending during a boom is not indicative of properly recognized future demand of resources.
Please note how I say current demand, and don't over project as to what my opinion about future demand is. I think there's a small, but reasonable chance that demand will sink for some reason or another in the next few years, and I think there's a pretty decent chance that in the next five years someone will come up with some way to make these things an order of magnitude or more more efficient, which would crash their electricity demands. But it's not a hypothetical "we need power in two years", or at least, not just that... it's we need more power now.
There's a big difference between "I may hypothetically need some more capacity later, I'd better go buy it now" and "I concretely need more capacity right now".
Earlier quoted context omitted.
GPU compute in datacenters has been a thing for at least 20 years. Many of the top500 have included significant GPU clusters for that long. There's nothing computationally special about AI compared to other workloads, and in fact it seems to lend itself to multiplexing quite efficiently - it's possible to process thousands of prompts for a negligable memory bandwidth increase over a single prompt. AI is still very ne…
> GPU compute in datacenters has been a thing for at least 20 years. Many of the top500 have included significant GPU clusters for that long. Of course they've been a thing, but for specialised situations, maybe rendering farms or backroom mining centers but it's disingenuous to claim that there's not an exponential growth in gpu useage.
Jest aside, the use of digital computation has exploded exponentially, for sure. But alongside that explosion, fueled by it and fueling it reciprocally, the cost (in energy and dollars) of each computation has plummeted exponentially.
Earlier quoted context omitted.
> Moore will continue to exponentially decrease costs over time as with all other workloads. There's absolutely no guarantee of this. The continuation of Moore's law is far from certain (NVIDIA think it's dead already).
> NVIDIA think it's dead already Perhaps that's what Jensen says publicly, but Nvidia's next generation chip contains more transistors than the last. And the one after that will too. Let me know when they align their $Trillions behind smaller less complex designs, then I'll believe that they think Moore's law is out of juice. Until then, they can sit with the group of people who've been vocally wrong about moore's la…
Earlier quoted context omitted.
> NVIDIA think it's dead already Perhaps that's what Jensen says publicly, but Nvidia's next generation chip contains more transistors than the last. And the one after that will too. Let me know when they align their $Trillions behind smaller less complex designs, then I'll believe that they think Moore's law is out of juice. Until then, they can sit with the group of people who've been vocally wrong about moore's la…
progress continues, but at far slower rates than they used to. nvidia has gained ~6x density in the past 9 years (1080 to 5090), while a doubling every 2 years would be >20x density in 9 years. the past 6 years (3090) are even worse with only a 3x of density
"The complexity for minimum component costs has increased at a rate of roughly a factor of two per year."
Density is one way in which industry has met this observation over decades. New processes (NMOS, CMOS, etc) is another. New packaging techniques (flip chip, BGA, etc). New substrates. There's no limit to process innovation.
Nvidia's also optimizing their designs for things other than minimum component cost. I.e. higher clock speeds, lower temperatures, lower power consumption, etc. It may seem like I'm picking a nit here, but such compromises are fundamental to the cost efficiency Moore was referencing.
All data I've seen, once fully considered, indicates that Moore's law is healthy and thriving.
Earlier quoted context omitted.
> GPU compute in datacenters has been a thing for at least 20 years. Many of the top500 have included significant GPU clusters for that long. Of course they've been a thing, but for specialised situations, maybe rendering farms or backroom mining centers but it's disingenuous to claim that there's not an exponential growth in gpu useage.
Of course they've been a thing, but for specialized situations, maybe calculating trajectories or breaking codes but it's disingenuous to claim that there's not an exponential growth in digital computer usage. Jest aside, the use of digital computation has exploded exponentially, for sure. But alongside that explosion, fueled by it and fueling it reciprocally, the cost (in energy and dollars) of each computation has…
It's been a while, but I don't recall any of the dotcom startups making deals with nuclear energy companies to buy out entire nuclear power stations: https://www.npr.org/2024/09/20/nx-s1-5120581/three-mile-isla... And that's just an example, there are many power-related deals of similar magnitude. The companies building out capacity certainly believe that AI is going to use as much power as we are told. We are told t…
Yes, big bets tell us something but they are not a crystal ball. Some of the same companies hired lots of people post-pandemic and then reversed. People who control enormous amounts of money can make risky bets that turn out to be wrong.
Earlier quoted context omitted.
Of course they've been a thing, but for specialized situations, maybe calculating trajectories or breaking codes but it's disingenuous to claim that there's not an exponential growth in digital computer usage. Jest aside, the use of digital computation has exploded exponentially, for sure. But alongside that explosion, fueled by it and fueling it reciprocally, the cost (in energy and dollars) of each computation has…
I really would like more of your data to show that, I think it would put this discussion to rest actually because I keep seeing articles that dispute it. At least older ones that ring bells, specifically https://epoch.ai/blog/trends-in-the-dollar-training-cost-of-...
Along with this lovely graph captioned: "Exponential growth of supercomputer performance per watt based on data from the Green500 list." (note the log scale): https://en.wikipedia.org/wiki/Performance_per_watt#/media/Fi...
From the section about GPU performance per watt, I'll quote:
"With modern GPUs, energy usage is an important constraint on the maximum computational capabilities that can be achieved. GPU designs are usually highly scalable, allowing the manufacturer to put multiple chips on the same video card, or to use multiple video cards that work in parallel. Peak performance of any system is essentially limited by the amount of power it can draw and the amount of heat it can dissipate. Consequently, performance per watt of a GPU design translates directly into peak performance of a system that uses that design."
Earlier quoted context omitted.
Yes, big bets tell us something but they are not a crystal ball. Some of the same companies hired lots of people post-pandemic and then reversed. People who control enormous amounts of money can make risky bets that turn out to be wrong.
There's a theory BigTech hired to hoard talent to inhibit competitiveness of rival companies.
