Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Performance per dollar is getting faster and cheaper

wafer.ai

Post reply on HN
Print view View original

131–140 of 151 posts

Re: Performance per dollar is getting faster and cheaper

#131

Post by minraws »

wongarsu wrote: post #105
embedding-shape wrote: post #89

Earlier quoted context omitted.

What type of company are you talking about here? Granted, nowadays I mostly interact with ML-adjacent companies but almost none would go "Hmm, hard to get nvidia hardware today, lets dump all expertise and knowledge of CUDA et al we have and start using AMD hardware until we can get nvidia", everyone would just wait or rent in the meantime.

Inference workloads are usually a lot less picky about the exact hardware than model training. At least in the cases I know of the models are trained on Nvidia hardware, then exported and run on a mix of Nvidia and AMD

At scale for inference it's almost non-existent for a data center company to go for AMD because they couldn't get or afford Nvidia atm.

They instead start the build out and plug in stuff they can, then take a loan or ask Nvidia to help fund it. (I am not joking)

I believe the case is if you can prove to Nvidia you can install and provide more Nvidia capacity they help out because more Capacity going online today is in the best interest of Nvidia.

Spot prices of Nvidia GPUs going up is not good news for Nvidia btw. The people renting Nvidia has the least amount of friction in moving off Nvidia, especially with AI tools you could build and get up to speed with AMD stack much sooner...

So if Nvidia is truly not an option and you entire company is not a bet on Nvidia then you will move off but only as a renter not as a buyer unless they truly can't fund Nvidia I suppose.

But again I repeat if you build a datacenter and provide good enough base Nvidia will help fund you to a mostly complete data center.

People might not like it but that's the reason Nvidia is so unreasonably dominant even now when otherwise given the scale of investments it might have been cheaper to look for alternatives.

This is why Nvidia doesn't like the China stack.

Re: Performance per dollar is getting faster and cheaper

#132

Post by minraws »

hassaanr wrote: post #41

While cool, quantization to FP4 is practically never lossless in actual use. A lot of providers are advertising high TPS on Kimi and GLM, but the models are functionally lobotomized and no longer close to frontier quality. Would love to see this not be true.

Doesn't Nvidia with their NVFP4 claim that it's lossless?

I haven't tested enough models Nvidia has converted to NVFP4 besides GLM 5.2 but it seemed fine to me.

My own luck has been hit or miss with it.

Re: Performance per dollar is getting faster and cheaper

#133

Post by quickthrowman »

ninjalanternshk wrote: post #107
fragmede wrote: post #93

Earlier quoted context omitted.

The problem these days is lack of nuance. It should seem entirely reasonable to be pro-datacenters-if-they're-done-right, but it feels like there are only two sides to any issue. Gas turbine whine noise isn't coming from the data center, it's being used to power the data center, but the camp is either pro data center or not, and fuck any nuance.

The problem is people keep trying to regulate businesses by name instead of by the effects they have. If we had regulations on noise, vibration, emissions, water use, electromagnetic radiation, whatever else, then it wouldn’t matter what people tried to build — if it fits within the guidelines great, otherwise back to the drawing board. Putting “data center” in your ordinances is as lazy and ineffective as putting “a…

> If we had regulations on noise, vibration, emissions, water use, electromagnetic radiation, whatever else, then it wouldn’t matter what people tried to build — if it fits within the guidelines great, otherwise back to the drawing board.

Sane jurisdictions do have regulations regarding these things. Not all jurisdictions are sane, some of them are run by people who sell out their residents.

Suburbs and cities around me all have noise regulations, my state has its own pollution regulations, and the local water utilities don’t hook up customers that stress the system. Unfortunately there are places like Texas, Tennessee, Louisiana, Mississippi that don’t give two shits about their citizens and let companies run temporary natural gas turbines permanently and all kinds of other nonsense.

Re: Performance per dollar is getting faster and cheaper

#134

Post by willis936 »

kingstnap wrote: post #21
minraws wrote: post #2

Can you folks add performance per watt as a metric to these comparisons, I honestly want to understand where AMD fits in the stack in terms of actual performance to dollars. I have had talks with companies wanting to build data centers outside of US and find it hard to source anything Nvidia in sufficient capacity and scale. If AMD is competitive performance per watt and roughly reliable in terms of software support…

A DGX B200 costs like ~$0.5 M and uses around 14 kW. If you plan to run it straight for 8 years 100% max usage thats around 1 GWhr. A gigawatt hour is a lot of energy but its not that much compared to the price of the actual machine. In Germany for example with its expensive energy thats about €100k worth, which spread over 8 years is pretty minor compared to the up front half mill. The real issue with high power con…

What they're really asking the authors is "can you not lie about performance cost and do proper accounting?". You can spin any story if you cherry pick your framing sufficiently. Stopping right at the silicon packaging boundary is as meaningless as it seems.

The article is highly qualified but the headline is not. If they are not making general statements then they shouldn't open with them.

Re: Performance per dollar is getting faster and cheaper

#136

Post by well_ackshually »

butvacuum wrote: post #65
psychoslave wrote: post #61

Earlier quoted context omitted.

And datacenters have impact on everything around them. If at the end of the day to result is a few more yachts and jets and, a lot more of miserable humans starving in ruined ecosystems, maybe that’s not the best go-to direction.

You say they have a large impact, but having lived somewhere with some of the largest data centers- they very much don't. At least not more so then any other structure that paves over greenery. love to debate actual discission points. pull up "datacenter dfw" on google maps for mine.

a constant low 60dB 20Hz hum in the background, 24/7 is as close as a a torture technique invented by the CIA as it can get.

Re: Performance per dollar is getting faster and cheaper

#137

Post by jfengel »

wongarsu wrote: post #102
__egb__ wrote: post #100

Earlier quoted context omitted.

Maybe the lack of nuance is due to learning, through decades of experience, that the assumption “it won’t be done right” can be baked in.

So people have a decades-long expectation that local government will fail them? This does sound plausible, but it's also pretty sad and not a sign of a healthy democracy

I'm hard pressed to think of anyone who believes that America has a healthy democracy. Even those most recently elected continually claim that democracy is under threat.
Post reply on HN