So... the headline is about performance per dollar per dollar?
Performance per dollar is getting faster and cheaper
121–130 of 151 posts
Re: Performance per dollar is getting faster and cheaper
#122The compute-in-memory and neuromorphic paradigms are likely to push this much, much farther over the next decade as more radical improvements make it out of the lab. Sooner or later it will involve new materials and new nano devices and providing multiple orders of magnitude better efficiency. And just scaling up existing things like MRAM.
Re: Performance per dollar is getting faster and cheaper
#123Earlier quoted context omitted.
The fact that people have lived and worked near data centres for decades and didn't even know what the term meant - let alone be adversely impacted by them - probably indicates they're broadly an non issue. All of a sudden out of nowhere, AI and data centres got intermingled by the media and now people seem to have big issues with them.
Though, the new data centers are not entirely the same. Increasing use of onsite gas turbines to generate power instead of using grid power changes their noise+air pollution profile.
afaik, it's only the so called "portable" generators openAI used to contravene noise and pollution regulations.
Re: Performance per dollar is getting faster and cheaper
#124Earlier quoted context omitted.
The fact that people have lived and worked near data centres for decades and didn't even know what the term meant - let alone be adversely impacted by them - probably indicates they're broadly an non issue. All of a sudden out of nowhere, AI and data centres got intermingled by the media and now people seem to have big issues with them.
Because the dynamics have shifted enormously inside the rack. 10 years ago, I was running 4 CPU servers with 48 cores and 128GB of RAM in 2U enclosures with a maximum power consumption of 500W or so. I was able to stick ~20 of them in a 42U rack, totaling 10kW. A data center full of these can be cooled with CRACs and hot/cold aisles without much problem. This is still too much for a bog-standard server colocation ope…
Why are you implying all datacenters are GPU farms? You can't retrofit that kind of power density into existing buildings.
You can stuff GPU servers into existing buildings- but even with significant upgrades you end up with a lot of empty space on the floor that can't be used.
Re: Performance per dollar is getting faster and cheaper
#125then why is token per dollar getting more expensive?
There are a limited number of these available in comparison to demand. I think people figured out that LLMs and VLMs can do real work that can replace a lot of humans. And for plenty of jobs, it's good enough to reduce already outsourced staff by 75-90% at a fraction of the cost.
Re: Performance per dollar is getting faster and cheaper
#126Earlier quoted context omitted.
Because the dynamics have shifted enormously inside the rack. 10 years ago, I was running 4 CPU servers with 48 cores and 128GB of RAM in 2U enclosures with a maximum power consumption of 500W or so. I was able to stick ~20 of them in a 42U rack, totaling 10kW. A data center full of these can be cooled with CRACs and hot/cold aisles without much problem. This is still too much for a bog-standard server colocation ope…
Why are you implying all datacenters are GPU farms? You can't retrofit that kind of power density into existing buildings. You can stuff GPU servers into existing buildings- but even with significant upgrades you end up with a lot of empty space on the floor that can't be used.
Two main reasons.
1. Article is about AI, so I have given the example for an AI datacenter.
2. In pure CPU datacenters, the power dynamics do not change much. I can add more servers to a single rack, but the rack power is again in the 30kW to 50kW range, so you're planning and building for the same power capacity.
> You can stuff GPU servers into existing buildings-
Yes.
> but even with significant upgrades you end up with a lot of empty space on the floor that can't be used.
Yes & No. It's not impossible to convert an old datacenter to support ~35KW/rack capacity, but it's not cheap, and you'll have more worries than holes, piping, building and power. Namely, can your floor handle that much weight to begin with?
Re: Performance per dollar is getting faster and cheaper
#127Earlier quoted context omitted.
> I have never seen a company use AMD outside of wafer and a couple others mostly in US. Worth remembering AMD basically "owns" (not literally) the hardware-side of things in video games consoles for good many years now, with no end in sight.
Because they have x86 CPU licenses.
AMD invented x86_64
Re: Performance per dollar is getting faster and cheaper
#128Can you folks add performance per watt as a metric to these comparisons, I honestly want to understand where AMD fits in the stack in terms of actual performance to dollars. I have had talks with companies wanting to build data centers outside of US and find it hard to source anything Nvidia in sufficient capacity and scale. If AMD is competitive performance per watt and roughly reliable in terms of software support…
A DGX B200 costs like ~$0.5 M and uses around 14 kW. If you plan to run it straight for 8 years 100% max usage thats around 1 GWhr. A gigawatt hour is a lot of energy but its not that much compared to the price of the actual machine. In Germany for example with its expensive energy thats about €100k worth, which spread over 8 years is pretty minor compared to the up front half mill. The real issue with high power con…
It's not even about the costs, getting enough power for a large datacenter is impractically hard in most of the world at a single location.
If it's efficient and the power costs of not just ongoing costs but the upfront setup is lower that makes a lot different scales of data centers practical, especially for inference which doesn't need massive super clusters.
You can't just fire up gas turbines everywhere like US Data centers are doing. I am not even sure if that's legal in US...
Note you have to plan for peak usage and a lot of stuff large scale data centers are insane infrastructure projects.
Nvidia is both supply and price constrainted, sure if you are willing to pay over 0.5M$ you might get some, but if you try to balance out price to costs by going slightly lower on the pole you realize just how much more expensive Nvidia truly feels like AMD has a lot of margin to under cut them if they want to.
Re: Performance per dollar is getting faster and cheaper
#129Can you folks add performance per watt as a metric to these comparisons, I honestly want to understand where AMD fits in the stack in terms of actual performance to dollars. I have had talks with companies wanting to build data centers outside of US and find it hard to source anything Nvidia in sufficient capacity and scale. If AMD is competitive performance per watt and roughly reliable in terms of software support…
> I have never seen a company use AMD outside of wafer and a couple others mostly in US. Worth remembering AMD basically "owns" (not literally) the hardware-side of things in video games consoles for good many years now, with no end in sight.
I was talking in the data center gpu context, EPYCs are pretty common in data centers these days.
I have a huge EPYC based data center like 200-300+km from my house on the outskirts of the city a few dozen miles from a IT industry tech park(place with lots of IT company offices).
Re: Performance per dollar is getting faster and cheaper
#130Can you folks add performance per watt as a metric to these comparisons, I honestly want to understand where AMD fits in the stack in terms of actual performance to dollars. I have had talks with companies wanting to build data centers outside of US and find it hard to source anything Nvidia in sufficient capacity and scale. If AMD is competitive performance per watt and roughly reliable in terms of software support…
> I have never seen a company use AMD Meta is using AMD: https://www.amd.com/en/newsroom/press-releases/2026-2-24-amd... And OpenAI: https://www.amd.com/en/newsroom/press-releases/2025-10-6-amd...
OpenAI maybe, but a few friends in Meta said they don't so dunno man. Seems sus atm.
But it's meta they can get a GW up of AMD in a year