Live data from Hacker News

Meta reuses old RAM in new servers with custom bridge chip

theregister.com

81–90 of 240 posts

Re: Meta reuses old RAM in new servers with custom bridge chip

#81

From the paper: "Our CXL solution achieves substantial gains for diverse workloads, including up to a 25% reduction in server count for disaggregated ML inference" How does using worse RAM result in 25% reduction of server count for given workloads?

CXL is adding "slow" RAM over PCIe, basically. Not replacing.

Re: Meta reuses old RAM in new servers with custom bridge chip

#82

I wonder if it would ever start to make sense to burn an AI model into ROM, replacing a large portion of an inference machine's RAM with ROM. (Probably not, since I'm sure those machines do dual-duty and run training when the inference workload slows down)

That's the idea behind Taalas (https://taalas.com), except as silicon rather than ROM. They run a demo at https://chatjimmy.ai/ which serves an old open weights model (Llama 3.1 8B) at something like 15,000 tokens per second.

Re: Meta reuses old RAM in new servers with custom bridge chip

#83
post #46

In the future, hardware is only for big companies to own. At least it seems we're heading that way.

Have you seen the talk: The Coming War on General Computation?

No, but I'm sure I'll agree with a lot of things in it.

Re: Meta reuses old RAM in new servers with custom bridge chip

#84
post #70

At the beginning of William Gibson’s Neuromancer , the protagonist is trying to sell 3 MB of RAM in underground markets. This is often cited as one of the ways the book has not aged well. But, looking at the direction of the memory market now… maybe we just haven’t gotten there yet.

It's sort of a cool idea. "Pre-RAM" without the tracking/AI integration so it can be used for clandestine activities in a dystopian future.

It's 3MB of "hot" RAM, IIRC. Makes sense.

Re: Meta reuses old RAM in new servers with custom bridge chip

#86
post #80

Earlier quoted context omitted.

> In the future, hardware is only for big companies to own. At least it seems we're heading that way. China is desperate to sell anything to... everyone. If there's a market, they'll eventually be there to fill it. It took them decades for cars, but now they did it. For RAM, CXMT went from 20 000 wafers per month to... 240 000 wafers per month in something like two years. And they're extending capacity massively now.…

Can you buy Chinese cars without the Multimedia/GPS computer and "phone home" system?

Maybe? But there's honestly not much market for this, as I'd guess less than 1% of the population cares about this. Particularly since they're already carrying a phone anyway.

Re: Meta reuses old RAM in new servers with custom bridge chip

#87
post #74

Earlier quoted context omitted.

observation that I've noticed recently: what's with wikipedia downsizing the hell out of images site wide? Every image I look at is garbage and I have to dig through multiple links to find the original.

I saw this with screenshots already for like 10-15 years now? It's some overly conservative policy to comply with fair use. I recall a page outlining that you should basically pick the minimum possible resolution that still allows you to distinguish the features of interest. I get why they do this, but it's really horrible from archiving and accessibility perspectives... And I would like to treat Wikipedia as one of…

I don’t think that’s applicable here?

It says the source photo was uploaded as original content by a user in 2011 as a 400x300 JPEG created on an iPhone 3GS per EXIF data, with copyright released as public domain.

There’s nothing to suggest it was downscaled in the log or copyright encumbered, it just looks like it’s old/small. I often click into Wikipedia/Wikimedia Commons images where the original is available as a super high resolution option in addition to various smaller thumbnails.

https://commons.wikimedia.org/wiki/File:IRAM13a.JPG

Re: Meta reuses old RAM in new servers with custom bridge chip

#88

At the beginning of William Gibson’s Neuromancer , the protagonist is trying to sell 3 MB of RAM in underground markets. This is often cited as one of the ways the book has not aged well. But, looking at the direction of the memory market now… maybe we just haven’t gotten there yet.

Early computer scientists were so optimistic. They beleives with a few kh of ram and a mhz of cpu they could do anything. Ai, consciousness, ml, language, text to speech. Now we spend gigs of ram on web forms. So gibson saying yeay 3MB of ram would probably be enough for a consciousness in cyber space, is very optimistic but fitting.

Re: Meta reuses old RAM in new servers with custom bridge chip

#89

At the beginning of William Gibson’s Neuromancer , the protagonist is trying to sell 3 MB of RAM in underground markets. This is often cited as one of the ways the book has not aged well. But, looking at the direction of the memory market now… maybe we just haven’t gotten there yet.

Memory in particular is something that I've reflected on more than once as having the most impressive gains in computing since I started paying attention to it (networking/USB too, but that doesn't make your computer "faster" in the same way).

I remember being able to borrow a computer from somewhere when Diablo II had just come out in 2000 which had a 450Mhz Pentium III and 64 MB of RAM. 64MB of RAM was probably mid-tier at the time, i.e. very much not a given. As I recall Diablo II recommended 64MB for single player and 128MB for multiplayer (or above 4 players or something).

The computer I'm writing this on has 64 GB of RAM, 1024 times as much. By comparison I have a 20-core Intel CPU with up-to 3GHz speed or somewhere around there, even pretending each core could run at that max speed simultaneously (which they can't), that's only 133-times as much CPU power.

Maybe the NVMe read times are as/more significant than memory size increase, but the metrics on them isn't quite as front and center on PC specs as memory and CPU.

Hard drive capacity similarly impressive as RAM in terms of size (was apparently 10-30GB in 2000), but I don't have a 10TB hard disk as I don't need one that big (1TB is plenty for me), so again it's not as impactful to me as memory.

Re: Meta reuses old RAM in new servers with custom bridge chip

#90
post #44

I have always wondered why there was never a big market[1] for "cheap PCI/PCI-X/PCI-e card you can stick a boatload of your old/surplus/n-generation old simms/dimms on and use as swap/slow memory/ram disk/etc". It's rare you can populate a motherboard with a full address space full of 'new' memory, and you can teach kernels to prefer some memory to others because of speed[2], so it seems like a no-brainer. I seem to…

I have always wondered why there was never a big market[1] for "cheap PCI/PCI-X/PCI-e card you can stick a boatload of your old/surplus/n-generation old simms/dimms on and use as swap/slow memory/ram disk/etc"

Reminds me of the days of JBOD arrays. Mac OS X had built-in support for it.

JBOR?

Post reply on HN