Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Performance per dollar is getting faster and cheaper

wafer.ai

Post reply on HN
Print view View original

91–100 of 151 posts

Re: Performance per dollar is getting faster and cheaper

#91

Post by hassaanr »

zozbot234 wrote: post #66
hassaanr wrote: post #41

While cool, quantization to FP4 is practically never lossless in actual use. A lot of providers are advertising high TPS on Kimi and GLM, but the models are functionally lobotomized and no longer close to frontier quality. Would love to see this not be true.

Kimi uses INT4 as its native format, there's no such thing as "better than 4-bit precision" for that model. This is in contrast with GLM for which 16-bit precision is native and 8-bit is in common use.

You’re right, but this poses a separate issue as the providers then do FP4 PTQ, which is quite lossy. Reduces the model size and optimizes for Blackwells at the (imo severe) cost of performance.

Re: Performance per dollar is getting faster and cheaper

#93

Post by fragmede »

Liftyee wrote: post #74
apublicfrog wrote: post #73

Earlier quoted context omitted.

The fact that people have lived and worked near data centres for decades and didn't even know what the term meant - let alone be adversely impacted by them - probably indicates they're broadly an non issue. All of a sudden out of nowhere, AI and data centres got intermingled by the media and now people seem to have big issues with them.

Though, the new data centers are not entirely the same. Increasing use of onsite gas turbines to generate power instead of using grid power changes their noise+air pollution profile.

The problem these days is lack of nuance. It should seem entirely reasonable to be pro-datacenters-if-they're-done-right, but it feels like there are only two sides to any issue. Gas turbine whine noise isn't coming from the data center, it's being used to power the data center, but the camp is either pro data center or not, and fuck any nuance.

Re: Performance per dollar is getting faster and cheaper

#95

Post by ozgrakkurt »

nxtfari wrote: post #32

I think we should make it illegal to not specify the quantization in the headline for these types of posts.

A nice filter is checking for the `.ai` in the end. It is very likely slop if you see that. Slop meaning low-effort/clickbait/shallow/useless/scam etc.

Re: Performance per dollar is getting faster and cheaper

#99

Post by Forgeties79 »

fragmede wrote: post #93
Liftyee wrote: post #74

Earlier quoted context omitted.

Though, the new data centers are not entirely the same. Increasing use of onsite gas turbines to generate power instead of using grid power changes their noise+air pollution profile.

The problem these days is lack of nuance. It should seem entirely reasonable to be pro-datacenters-if-they're-done-right, but it feels like there are only two sides to any issue. Gas turbine whine noise isn't coming from the data center, it's being used to power the data center, but the camp is either pro data center or not, and fuck any nuance.

Because the reality is while we all debate the nuances companies just do whatever they want, and it’s usually whatever offloads the most issues to the public because it saves them more money.

Re: Performance per dollar is getting faster and cheaper

#100

Post by __egb__ »

fragmede wrote: post #93
Liftyee wrote: post #74

Earlier quoted context omitted.

Though, the new data centers are not entirely the same. Increasing use of onsite gas turbines to generate power instead of using grid power changes their noise+air pollution profile.

The problem these days is lack of nuance. It should seem entirely reasonable to be pro-datacenters-if-they're-done-right, but it feels like there are only two sides to any issue. Gas turbine whine noise isn't coming from the data center, it's being used to power the data center, but the camp is either pro data center or not, and fuck any nuance.

Maybe the lack of nuance is due to learning, through decades of experience, that the assumption “it won’t be done right” can be baked in.
Post reply on HN