Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The RAM shortage could last years

theverge.com

Post reply on HN
Print view View original

321–330 of 536 posts

Re: The RAM shortage could last years

#321

Post by paganel »

energy123 wrote: post #306
paganel wrote: post #291

Earlier quoted context omitted.

Europe still has coal, lots of it, not using it is a political choice and a self-inflicted wound.

Last year, China's coal use decreased, while China installed 300x more renewables than nuclear. Coal and nuclear aren't cost competitive with renewables, either in a free market or a technocratic top-down economy. Coal and gas still maintain a valid niche of firming intermittency. But that niche is temporary and shrinking.

> either in a free market o

Then why all the anti-coal mining diktats coming down from Brussels?

Re: The RAM shortage could last years

#322

Post by Nesco »

Ardon wrote: post #182
kev009 wrote: post #157

Earlier quoted context omitted.

Is the DRAM industry really capitalist? Focusing on just the Korean parties, it functions like a command economy. I would say the same about most high end semi-conductor manufacturing, TSMC, Intel, ASML are being commanded and driven by nation-state level decision making. Right now the command is to focus on high wattage centralized AI systems at the expense of everything else.

No one at high levels is capitalist, in ideology or action. An ideological capitalist would be in favor of competition, but these people disdain it and collude regularly. The only 'capitalist' actions they take are by accident, the real goal is as much power/money as possible as fast as possible. We don't even expect companies to plan long-term anymore, it's just moving wealth as fast as possible. That isn't really a…

Liberalism (in the traditional economic sens) likes competition. Capitalism is a mode of production, and capitalists notoriously don’t like competition when they are the incumbents

Re: The RAM shortage could last years

#323

Post by Nesco »

LNSY wrote: post #229
torginus wrote: post #42

Earlier quoted context omitted.

The Radeon VII came out in 2019 as a $700 consumer GPU with an 1TB/s HBM2 memory subsystem which is more than any consumer GPU you can get today, including the high-end ones afaik. At that point in time, there was a whole lineup of AMD GPUs with HBM going down into the midrange. If they could make this stuff and sell it to regular people a decade ago for very palatable prices, why do they come up with the idea that t…

[flagged]

What do you mean exactly by “Epstein People”? I thoroughly hope it’s not the 4chan definition of it

Re: The RAM shortage could last years

#324

Post by PunchyHamster »

stuxnet79 wrote: post #11

Ok so Samsung, SK Hynix and Micron do not have the capacity to meet demand. Also, what little capacity they do have they are allocating to HBM over DRAM. Based on my limited knowledge HBM can not be easily repurposed for consumer electronics. Translation: main street is cooked for the next 3-4 years. It doesn't stop there though. OpenAI is currently mired in a capital crunch. Their last round just about sucked all th…

> Based on my limited knowledge HBM can not be easily repurposed for consumer electronics. Translation: main street is cooked for the next 3-4 years.

It's worse. HBM have lower yields so they are essentially making less GB per wafer too

Re: The RAM shortage could last years

#325

Post by NooneAtAll3 »

fouc wrote: post #3

I'm a bit surprised the article makes no mention of Google's TurboQuant[0] introduced 26 days prior. Given that TurboQuant results in a 6x reduction in memory usage for KV caches and up to 8x boost in speed, this optimization is already showing up in llama.cpp, enabling significantly bigger contexts without having to run a smaller model to fit it all in memory. Some people thought it might significantly improve the R…

> 6x reduction in memory usage for KV caches and up to 8x boost in speed

mind that you're quoting marketing material that's largely based on unfair baseline testing (like comparing 4 bit vs 32 bit to get "8x speed")

https://www.youtube.com/watch?v=haoAI2lIZ74

Re: The RAM shortage could last years

#326

Post by PunchyHamster »

crote wrote: post #287
trvz wrote: post #122

Earlier quoted context omitted.

And all of these are practically irrelevant for AI data centers.

Do AI data centers not need internet connectivity anymore? The value of an IX isn't just in the IX itself, but also in the presence of hundreds of parties for direct peering, and excellent connectivity to the rest of the world. It makes a lot of sense to build your DC near one - even if you have no intention of actually participating in the IX itself.

> Do AI data centers not need internet connectivity anymore?

They don't need entire IX worth of connectivity. You're sending mostly text back and forth and any media is in far lower volume than even normal far less dense DC would generate, all the major traffic is inside the AI DC.

All it needs is fiber to nearest IX

Re: The RAM shortage could last years

#327

Post by PunchyHamster »

CamperBob2 wrote: post #237
formerly_proven wrote: post #102

Earlier quoted context omitted.

DRAM and to a lesser degree storage are notorious for their feast and famine cycles (Well that and collusion)

The problem in this case seems to have sprung from a lack of collusion. Altman reportedly approached Samsung and SK independently to strike deals for a large chunk of both companies' production. Neither party apparently knew he was negotiating with the other. If they had actually been communicating or colluding with each other, they would have put the screws to him, making it harder for OpenAI to assert control over…

> Failing that, you'd like to think a regulatory agency somewhere would step in to keep a single player from hosing everybody else, but...

Up until AI there weren't really players being able to gobble 40% of the market so nobody was looking.

Re: The RAM shortage could last years

#328

Post by PunchyHamster »

whatever1 wrote: post #147
bombcar wrote: post #79

Earlier quoted context omitted.

Don’t the memory makers always get left holding the bag? I feel this has happened at least three times before.

All of the capital intensive businesses face this issue. Chemicals, Shipping, Semiconductors etc. You get market signals that the demand is there, you acquire the necessary capital, you spend 5 years to build capacity, but guess what, 5 other market players did the same thing. So now you are doomed, because the market is flooded and you have low cash flow since you need to drop prices to compete for pennies. Now you…

The sheer fucking blazing ignorance of this comment

> Capitalists claim that this is optimal.

Compared to starving under communism coz someone at top got the number wrong, yes. And it only really happens when there are massive, unpredictable market movements and governments not doing their job. Govt should look at the whole thing and just say "no", blame them.

No market system self regulates well enough, and it's government job to file down the edge cases like this. But the revolution happened in country which has two utterly incompetent parties, both in pockets of billionaires, fighting for power, and the clowns from one that won last battle use AI to smokescreen the economic growth their actions cratered

Re: The RAM shortage could last years

#329

Post by PunchyHamster »

jandrewrogers wrote: post #191
whatever1 wrote: post #147

Earlier quoted context omitted.

All of the capital intensive businesses face this issue. Chemicals, Shipping, Semiconductors etc. You get market signals that the demand is there, you acquire the necessary capital, you spend 5 years to build capacity, but guess what, 5 other market players did the same thing. So now you are doomed, because the market is flooded and you have low cash flow since you need to drop prices to compete for pennies. Now you…

Forecasting demand 5 years into the future is intrinsically highly unreliable. It doesn’t matter if it is capitalism or a command economy. The bet is always going to be risky and someone will have to pay for that risk. At least with capitalism you have many different people with different perspectives on the risk making independent bets. That mitigates the more extreme negative outcomes.

Also it is government job to regulate. Monopolies should be busted, and behaviour like that should be culled. But US govt is full on AI to mask them cratering their own economy.

Re: The RAM shortage could last years

#330

Post by PunchyHamster »

overfeed wrote: post #261
trvz wrote: post #125

Earlier quoted context omitted.

Those financiers can’t risk not being involved in a company with even just a slight potential for AGI.

> Those financiers can’t risk not being involved in a company with even just a slight potential for AGI. Do recent actions of Open AI give you the impression of a company that believes it is about to attain AGI imminently?

All it matters that gullible investors do.

Hell, all it matters to investors is not being left holding the bag in the end so they don't even need to believe it

Post reply on HN