Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Meta reuses old RAM in new servers with custom bridge chip

theregister.com

Post reply on HN
Print view View original

21–30 of 240 posts

Re: Meta reuses old RAM in new servers with custom bridge chip

#21

Post by joha4270 »

rcxdude wrote: post #16
torginus wrote: post #10

With regards to RAM price I never understood the following: A 16GB RAM stick has 16*8=128 billion bits, with 1 transistor per bit, thats still 128B, yet its supposed to cost like $60 before the price hikes? In contrast, a 5090 GPU was $2000 (true it has RAM, but you're paying for the GPU ASIC really, I guess the rest of the GPU was less than $500), it had 93B transistors. GPU transistors are smaller due to the more a…

Chip fabrication processes are not fungible: GPUs and CPUs might be made on roughly the same process, but DRAM is not (flash is a different process again, as is power electronics, analog electronics, MEMS, etc. And even within those broader categories there are different variations). While there are some overlaps in machines and techniques, a fab set up for one cannot generally switch to the other, and the economics…

You're not the first person to say so (and I don't mean to dispute it), but I have never been able to find a clear answer for /why/ those processes are incompatible.

Is it built in different silicon, is it physical steps that's incompatible (ie its actually incompatible), is it different physical preparations that needs to be made (making it economically infeasible to combine)

I cannot help but wonder, even if the answer doesn't change anything in my life.

Re: Meta reuses old RAM in new servers with custom bridge chip

#22

Post by dofm »

rock_artist wrote: post #3

The interesting part of this "RAM crisis" is similar to other fields where a problem results multiple parties looking for alternative solutions. This yields for exciting ideas or workarounds that might result a post-crisis memory boom (hopefully) also for local machines. 1. Lowest, Apple is evaluating new Chinese manufacturer which means change of supply demand if indeed it has reasonable QA. ( https://www.ft.com/con…

Necessity is the mother of invention, after all. (One of the oldest abstract concepts in intellectual thought, I suspect.)

There is a tight resource starvation/motivation loop — the demand put on RAM and SSD and GPUs by the largest frontier models is a direct motivation to make smaller LLMs. Like an evolutionary pressure making animals smaller and more food-efficient.

These smaller models, once successful, are still likely to consume more RAM and SSD and GPUs than any other application short of high quality video processing itself (the smaller LLMs and higher end video processing seem to have about the same needs). But the resources would distribute through the market more traditionally, leading to less insane cycles.

So it seems to me that the way out of the RAM/SSD price cycle crisis that manufacturers are in — where the price fluctuates between high and low due to supply constraints and then oversupply from new production capacity - is for them to fund research into smaller LLMs. They'll still sell essentially the same amount of product. Maybe more.

Re: Meta reuses old RAM in new servers with custom bridge chip

#23

Post by HumblyTossed »

rock_artist wrote: post #3

The interesting part of this "RAM crisis" is similar to other fields where a problem results multiple parties looking for alternative solutions. This yields for exciting ideas or workarounds that might result a post-crisis memory boom (hopefully) also for local machines. 1. Lowest, Apple is evaluating new Chinese manufacturer which means change of supply demand if indeed it has reasonable QA. ( https://www.ft.com/con…

I would love it if we started designing software with hardware constraints in mind again.

Re: Meta reuses old RAM in new servers with custom bridge chip

#24

Post by WmWsjA6B29B4nfk »

torginus wrote: post #10

With regards to RAM price I never understood the following: A 16GB RAM stick has 16*8=128 billion bits, with 1 transistor per bit, thats still 128B, yet its supposed to cost like $60 before the price hikes? In contrast, a 5090 GPU was $2000 (true it has RAM, but you're paying for the GPU ASIC really, I guess the rest of the GPU was less than $500), it had 93B transistors. GPU transistors are smaller due to the more a…

On top of everything said, 5090 die size is 10x than typical DDR5 die size. One RAM module is 8-16 dies, so you do get more silicon in the end, but larger dies are extremely expensive to produce due to sharply decreasing yields.

Re: Meta reuses old RAM in new servers with custom bridge chip

#25

Post by keanebean86 »

jzb wrote: post #8

It’d be nice if there were a consumer version of this. I have plenty of old RAM.

Gigabyte had a ram disk addin card years ago. Not exactly the same but since it's presented as a storage device you could use it as OS swap space.

https://en.wikipedia.org/wiki/I-RAM

Re: Meta reuses old RAM in new servers with custom bridge chip

#26

Post by NDlurker »

herodoturtle wrote: post #17
pjc50 wrote: post #7

Earlier quoted context omitted.

Source paper linked is https://aisystemcodesign.github.io/papers/isca26/vistara_cam... From a quick skim, you could think of this as roughly equivalent to shoving a large amount of DDR4 on a PCIe card and using it as a swap space. It's more sophisticated (see CXL protocol), but that gives you an idea of the tradeoffs. It seems there is some OS-level support for moving hot/cold pages between the main fast DRAM and the…

> and contains significant embedded carbon emissions Hi - thanks for the insightful comment - could you please expand on the above? Genuinely curious :)

Reduce, reuse, recycle

Re: Meta reuses old RAM in new servers with custom bridge chip

#27

Post by jahnu »

joha4270 wrote: post #21
rcxdude wrote: post #16

Earlier quoted context omitted.

Chip fabrication processes are not fungible: GPUs and CPUs might be made on roughly the same process, but DRAM is not (flash is a different process again, as is power electronics, analog electronics, MEMS, etc. And even within those broader categories there are different variations). While there are some overlaps in machines and techniques, a fab set up for one cannot generally switch to the other, and the economics…

You're not the first person to say so (and I don't mean to dispute it), but I have never been able to find a clear answer for /why/ those processes are incompatible. Is it built in different silicon, is it physical steps that's incompatible (ie its actually incompatible), is it different physical preparations that needs to be made (making it economically infeasible to combine) I cannot help but wonder, even if the an…

https://www.reddit.com/r/Semiconductors/comments/r1dqmw/how_...

A Reddit user explains a bit here.

Re: Meta reuses old RAM in new servers with custom bridge chip

#29

Post by wolfi1 »

joha4270 wrote: post #21
rcxdude wrote: post #16

Earlier quoted context omitted.

Chip fabrication processes are not fungible: GPUs and CPUs might be made on roughly the same process, but DRAM is not (flash is a different process again, as is power electronics, analog electronics, MEMS, etc. And even within those broader categories there are different variations). While there are some overlaps in machines and techniques, a fab set up for one cannot generally switch to the other, and the economics…

You're not the first person to say so (and I don't mean to dispute it), but I have never been able to find a clear answer for /why/ those processes are incompatible. Is it built in different silicon, is it physical steps that's incompatible (ie its actually incompatible), is it different physical preparations that needs to be made (making it economically infeasible to combine) I cannot help but wonder, even if the an…

from my basic understanding, memory is much easier to produce then logic chips like GPUs and CPUs, they don't need that many photolithographic layers. while it could be possible to produce memory in fabs for CPUs (though not really desirable in regard to costs) the other way round is more difficult

Re: Meta reuses old RAM in new servers with custom bridge chip

#30

Post by crote »

joha4270 wrote: post #21
rcxdude wrote: post #16

Earlier quoted context omitted.

Chip fabrication processes are not fungible: GPUs and CPUs might be made on roughly the same process, but DRAM is not (flash is a different process again, as is power electronics, analog electronics, MEMS, etc. And even within those broader categories there are different variations). While there are some overlaps in machines and techniques, a fab set up for one cannot generally switch to the other, and the economics…

You're not the first person to say so (and I don't mean to dispute it), but I have never been able to find a clear answer for /why/ those processes are incompatible. Is it built in different silicon, is it physical steps that's incompatible (ie its actually incompatible), is it different physical preparations that needs to be made (making it economically infeasible to combine) I cannot help but wonder, even if the an…

The physical structure is completely different. Just compare DRAM ([0]) with compute ([1]). As a result, the production process is completely different.

If you want to know more, the Asianometry youtube channel has some fairly good deep dives, such as [2] going through a decent bunch of the 45nm production process, or [3] doing the same for (early) DRAM.

[0]: https://www.youtube.com/watch?v=Bln-v9LmZ3E

[1]: https://i1.wp.com/semiengineering.com/wp-content/uploads/201...

[2]: https://www.youtube.com/watch?v=zUgy29h0alM

[3]: https://www.youtube.com/watch?v=uPualBNf1nM

Post reply on HN