Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The RAM shortage comes for us all

jeffgeerling.com

Post reply on HN
Print view View original

181–190 of 416 posts

Re: The RAM shortage comes for us all

#181

Post by jandrese »

hodgehog11 wrote: post #60
Loic wrote: post #13

I think the OpenAI deal to lock wafers was a wonderful coup. OpenAI is more and more losing ground against the regularity[0] of the improvements coming from Anthropic, Google and even the open weights models. By creating a chock point at the hardware level, OpenAI can prevent the competition from increasing their reach because of the lack of hardware. [0]: For me this is really an important part of working with Claud…

I don't see this working for Google though, since they make their own custom hardware in the form of the TPUs. Unless those designs include components that are also susceptible?

That was why OpenAI went after the wafers, not the finished products. By buying up the supply of the raw materials they bottleneck everybody, even unrelated fields. It's the kind of move that requires a true asshole to pull off, knowing it will give your company an advantage but screw up life for literally billions of people at the same time.

Re: The RAM shortage comes for us all

#182

Post by bombcar »

oskarkk wrote: post #177
fullstop wrote: post #155

Earlier quoted context omitted.

~2.8x for DDR4, ~3.6x for DDR5. DDR5 is still being made, though, so it will be interesting to see how it changes in the future. Either way, it's going to be a long few years at the least.

Unless the AI bubble pops.

One possible outcome is the remaining memory manufacturers have dedicated all their capacity for AI and when the bubble pops, they lose their customer and they go out of business too.

Re: The RAM shortage comes for us all

#183

Post by thedangler »

MPSimmons wrote: post #89

This reminds me of the recent LaurieWired video presenting a hypothetical of, "what if we stopped making CPUs": https://www.youtube.com/watch?v=L2OJFqs8bUk Spoiler, but the answer is basically that old hardware rules the day because it lasts longer and is more reliable of timespans of decades. DDR5 32GB is currently going for ~$330 on Amazon DDR4 32GB is currently going for ~$130 on Amazon DDR3 32GB is currently goin…

Nice, My current PC uses DDR4 Time to dust off my 2012 PC and put Linux on it.

Re: The RAM shortage comes for us all

#184

Post by fullstop »

malfist wrote: post #92
hnuser123456 wrote: post #81

Earlier quoted context omitted.

Sure, but if the price is being inflated by inflated demand, then the suppliers will just build more factories until they hit a new, higher optimal production level, and prices will come back down, and eventually process improvements will lead to price-per-GB resuming its overall downtrend.

Micron has said they're not scaling up production. Presumably they're afraid of being left holding the bag when the bubble does pop

Why are they building a foundry in Idaho?

https://www.micron.com/us-expansion/id

Re: The RAM shortage comes for us all

#185

Post by georgeburdell »

fullstop wrote: post #161
phil21 wrote: post #156

Earlier quoted context omitted.

Currently trying to source a large amount of DDR4 to upgrade a 3 year old fleet of servers at a very unfortunate time. It's very difficult to source in quantity, and is going up in price more or less daily at this point. Vendor quotes are good for hours, not days when you can find it.

We're looking to buy 1TB of DDR5 for a large database server. I'm pretty sure that we're just going to pay the premium and move on.

And that’s why everybody’s B2B these days. The decision-making people at companies are not spending their own money

Re: The RAM shortage comes for us all

#186

Post by mebassett »

lysace wrote: post #145
Loic wrote: post #13

I think the OpenAI deal to lock wafers was a wonderful coup. OpenAI is more and more losing ground against the regularity[0] of the improvements coming from Anthropic, Google and even the open weights models. By creating a chock point at the hardware level, OpenAI can prevent the competition from increasing their reach because of the lack of hardware. [0]: For me this is really an important part of working with Claud…

Please explain to me like I am five: Why does OpenAI need so much RAM? 2024 production was (according to openai/chatgpt) 120 billion gigabytes. With 8 billion humans that's about 15 GB per person.

large language models are large and must be loaded into memory to train or to use for inference if we want to keep them fast. older models like gpt3 have around 175 billion parameters. at float32s that comes out to something like 700GB of memory. newer models are even larger. and openai wants to run them as consumer web services.

Re: The RAM shortage comes for us all

#187

Post by fullstop »

pja wrote: post #178
seanalltogether wrote: post #135

I wonder if these RAM shortages are going to cause the Steam Machine to be dead on arrival. Valve is probably not a big enough player to have secured production guarantees like Sony or Nintendo would have. If they try to launch with a price tag over $750, they're probably not going sell a lot.

Yeah, I think (sadly) this kills the Steam Machine in the short term if the competition is current consoles. At least until the supply contracts Sony & Microsoft have signed come up for renewal, at which point they’re going to be getting the short end of the RAM stick too. In the short term the RAM shortage is going to kill homebrew PC building & small PC builders stone dead - prebuilts from the larger suppliers will…

> At least until the supply contracts Sony & Microsoft have signed come up for renewal, at which point they’re going to be getting the short end of the RAM stick too.

Allegedly Sony has an agreement for a number of years, but Microsoft does not: https://thegamepost.com/leaker-xbox-series-prices-increase-r...

Re: The RAM shortage comes for us all

#188

Post by thisrobot »

hodgehog11 wrote: post #126
Grosvenor wrote: post #59

Earlier quoted context omitted.

Could this generate pressure to produce less memory hungry models?

There has always been pressure to do so, but there are fundamental bottlenecks in performance when it comes to model size. What I can think of is that there may be a push toward training for exclusively search-based rewards so that the model isn't required to compress a large proportion of the internet into their weights. But this is likely to be much slower and come with initial performance costs that frontier model…

I wonder if this maintains the natural language capabilities which are what LLM's magic to me. There is a probably some middle ground, but not having to know what expressions, or idiomatic speech an LLM will understand is really powerful from a user experience point of view.

Re: The RAM shortage comes for us all

#189

Post by Forgeties79 »

fithisux wrote: post #10

32GB should be more than enough. You can go 16GB if you go native and throw some assembly in the mix. Use old school scripting languages. Debloat browsers. It has been long delayed.

Sure but 32GB DDR5 ram has just jumped from ~$100 to $300+ in a flash. The 2x16GB I have in my recent build went from $105 for the pair to $250 each. $500 total!

SSD’s are also up. Hell I am seeing refurbished enterprise HDD’s at 2x right now. It’s sharp increases basically across the board except for CPU’s/GPU’s.

Every PC build basically just cranked up $400-$600 easily, and that’s not accounting for the impact of inflation over the last few years weakening everyone’s wallets. The $1600 machine I spec’d out for my buddy 5 weeks ago to buy parts for this Black Friday now runs $2k even.

Post reply on HN