Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The RAM shortage comes for us all

jeffgeerling.com

Post reply on HN
Print view View original

211–220 of 416 posts

Re: The RAM shortage comes for us all

#211

Post by nish__ »

squigz wrote: post #169
School-Cotton wrote: post #95

Earlier quoted context omitted.

China cannot buy ASML machines. All advanced semiconductor manufacturing in China is done with stockpiled ASML machines from before the ban.

As someone who knows next to nothing about this space, why can China not build their own machines? Is ASML the only company making those machines? If so, why? Is it a matter of patents, or is the knowledge required for this so specialized only they've built it up?

Yes. ASML is the only company making these machines. And both, they own thousands of patents and are also the only ones with the institutional knowledge required to build them anyway.

Re: The RAM shortage comes for us all

#213

Post by txdv »

jsheard wrote: post #2

I wonder if Apple will budge. The margins on their RAM upgrades were so ludicrous before that they're probably still RAM-profitable even without raising their prices, but do they want to give up those fat margins?

on one hand they are loosing profit, on the other hand they are gaining on market share. They will probably wait a short while to assess how much they are willing to sacrifice profits for market share

Re: The RAM shortage comes for us all

#216

Post by q3k »

nish__ wrote: post #53
bhhaskin wrote: post #30

Earlier quoted context omitted.

Only if you put up the 10 billion dollars.

Machines are less than 400 million.

You're just talking about a lithography machine. Patterning is one step out of thousands in a modern process (albeit an important one). There's plenty more stuff needed for a production line, this isn't a 3D printer but for chips. And that's just for the FEOL stuff, then you still need to do BEOL :). And packaging. And testing (accelerated/environmental, too). And failure analysis. And...

Also, you know, there's a whole process you'll need to develop. So prepare to be not making money (but spending tons of it on running the lines) until you have a well tested PDK.

Re: The RAM shortage comes for us all

#217

Post by shadowpho »

nish__ wrote: post #171
Reason077 wrote: post #76

Earlier quoted context omitted.

A dozen or so well-resourced tech titans in China are no doubt asking themselves this same question right now. Of course, it takes quite some time for a fab to go from an idea to mass production. Even in China. Expect prices to drop 2-3 years from now when all the new capacity comes online?

My napkin math: According to my research, these machines can etch around 150 wafers per hour and each wafer can fit around 50 top-of-the-line GPUs. This means we can produce around 7500 AI chips per hour. Sell them for $1k a piece. That's $7.5 million per hour in revenue. Run the thing for 3 days and we recover costs. I'm sure there's more involved but that sounds like a pretty good ROI to me.

What about the $10b to build the facility (including clean air/water/chemicals/etc)?

Re: The RAM shortage comes for us all

#218

Post by dist-epoch »

hathawsh wrote: post #111

The article suggests that because the power and cooling are customized, it would take a ton of effort to run the new AI servers in a home environment, but I'm skeptical of that. Home-level power and cooling are not difficult these days. I think when the next generation of AI hardware comes out (in 3-5 years), there will be a large supply of used AI hardware that we'll probably be able to repurpose. Maybe we'll sell t…

A single server is 20 KW. A rack is 200 KW.

These are not the old CPU servers of yesterday.

Re: The RAM shortage comes for us all

#219

Post by jiggawatts »

hodgehog11 wrote: post #126
Grosvenor wrote: post #59

Earlier quoted context omitted.

Could this generate pressure to produce less memory hungry models?

There has always been pressure to do so, but there are fundamental bottlenecks in performance when it comes to model size. What I can think of is that there may be a push toward training for exclusively search-based rewards so that the model isn't required to compress a large proportion of the internet into their weights. But this is likely to be much slower and come with initial performance costs that frontier model…

> exclusively search-based rewards so that the model isn't required to compress a large proportion of the internet into their weights.

That just gave me an idea! I wonder how useful (and for what) a model would be if it was trained using a two-phase approach:

1) Put the training data through an embedding model to create a giant vector index of the entire Internet.

2) Train a transformer LLM but instead only utilising its weights, it can also do lookups against the index.

Its like a MoE where one (or more) of the experts is a fuzzy google search.

The best thing is that adding up-to-date knowledge won’t require retraining the entire model!

Re: The RAM shortage comes for us all

#220

Post by xboxnolifes »

MPSimmons wrote: post #89

This reminds me of the recent LaurieWired video presenting a hypothetical of, "what if we stopped making CPUs": https://www.youtube.com/watch?v=L2OJFqs8bUk Spoiler, but the answer is basically that old hardware rules the day because it lasts longer and is more reliable of timespans of decades. DDR5 32GB is currently going for ~$330 on Amazon DDR4 32GB is currently going for ~$130 on Amazon DDR3 32GB is currently goin…

Unfortunately, older RAM also means an older motherboard, which also means older socket and older CPUs. It works, but it's not usually a drop in replacement.
Post reply on HN