Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

RAM now represents 35 percent of bill of materials for HP PCs

arstechnica.com

Post reply on HN
Print view View original

211–220 of 355 posts

Re: RAM now represents 35 percent of bill of materials for HP PCs

#211

Post by kasabali »

alephnerd wrote: post #69
Ygg2 wrote: post #63

Earlier quoted context omitted.

Then those countries that didn't will have an advantage at selling their electronics to the world. Or their consumers will enjoy cheap PC part prices. With possible gray zone re-export market. Of course we could see retreat from global markets to mercantilism, but that has yet to fully happen.

Or China could stop antagonizing blocs like the EU through actions like solidifying ties with Russia [0][1][2], imposing rare earth export restrictions on the EU [3], and undermining EU institutions [4]. [0] - https://www.reuters.com/world/asia-pacific/xi-putin-hail-tie... [1] - https://www.reuters.com/world/china/chinas-president-xi-meet... [2] - https://www.reuters.com/world/china/china-calls-closer-defen... [3] -…

> stop antagonizing blocs like the EU

Who antagonized who first, again?

Re: RAM now represents 35 percent of bill of materials for HP PCs

#212

Post by jacquesm »

yread wrote: post #128
jacquesm wrote: post #78

Earlier quoted context omitted.

> And then memory expanded so much that all kinds of “optimal” patterns for programming just become nearly irrelevant. I don't think that ever happened. Using relatively sparse amount of memory turns into better cache management which in turn usually improves performance drastically. And in embedded stuff being good with memory management can make the difference between 'works' and 'fail'.

When was the last time you used mergesort because you had to?

Coincidentially, last night, and I'm not pulling your leg! But to be fair that's the first time in much more than a decade. I don't normally work with such huge files and this was one very rare exception. I also nearly crashed my machine by triggering the OOM killer after naively typing 'vi file' without first checking how large it had become. I'm working on a project that I probably should run on a more serious machine but I don't feel like moving my whole work environment from the laptop that I normally use.

Re: RAM now represents 35 percent of bill of materials for HP PCs

#213

Post by KeplerBoy »

makapuf wrote: post #116
fulafel wrote: post #83

Earlier quoted context omitted.

You're right in terms of fitting your program to memory, so that it can run in the first place. But in performance work, the relative speed of RAM relative to computation has dropped such that it's a common wisdom to treat today's cache as RAM of old (and today's RAM as disk of old, etc). In software performance work it's been all about hitting the cache for a long time. LLMs aren't too amenable to caching though.

AFAIK, you can't explicitly allocate cache like you allocate RAM however. A bit like if you could only work on files and ram was used for cache. Maybe I am mistaken ? (Edit: typo)

You can in CUDA. You can have shared memory which is basically L1 cache you have full control over. It's called shared memory because all threads within a block (which reside on a common SM) have fast access to it. The downside: you now have less regular L1 cache.

Re: RAM now represents 35 percent of bill of materials for HP PCs

#214

Post by dahcryn »

travisgriggs wrote: post #50

I had my formative years in programming when memory usage was something you still worried about as a programmer. And then memory expanded so much that all kinds of “optimal” patterns for programming just become nearly irrelevant. Will we start to actually consider this in software solutions again as a result?

I've actively started to use outlook and teams through chrome to free up some of my ram, easily saves 3-4gb. It's gotten ridiculous how much ram basic tools are using, leaving nothing for doing actually real work

Re: RAM now represents 35 percent of bill of materials for HP PCs

#216

Post by zarzavat »

travisgriggs wrote: post #50

I had my formative years in programming when memory usage was something you still worried about as a programmer. And then memory expanded so much that all kinds of “optimal” patterns for programming just become nearly irrelevant. Will we start to actually consider this in software solutions again as a result?

RAM didn't get more expensive to produce. It just got more desirable. The prices will come down again when supply responds. It may take some time, but it will happen eventually.

Re: RAM now represents 35 percent of bill of materials for HP PCs

#217

Post by zozbot234 »

crakenzak wrote: post #155
lou1306 wrote: post #129

Earlier quoted context omitted.

You can't explicitly allocate cache, but you can lay things out in memory to minimize cache misses.

Out of curiosity, why has there not been a slight paradigm shift in modern system programming languages to expose more control over the caches?

This is not applicable to most programming scenarios since the cache gets trashed unpredictably during context switches (including the user-level task switches involved in cooperative async patterns). It's not a true scratchpad storage, and turning it into one would slow down context switches a lot since the scratchpad would be processor state. Maybe this can be revisited once even low-end computers have so many hardware cores/threads that context switches become so rare that the overhead is not a big deal. But we are very far from anything of the sort.

Re: RAM now represents 35 percent of bill of materials for HP PCs

#218

Post by zozbot234 »

seanmcdirmid wrote: post #111
fulafel wrote: post #83

Earlier quoted context omitted.

You're right in terms of fitting your program to memory, so that it can run in the first place. But in performance work, the relative speed of RAM relative to computation has dropped such that it's a common wisdom to treat today's cache as RAM of old (and today's RAM as disk of old, etc). In software performance work it's been all about hitting the cache for a long time. LLMs aren't too amenable to caching though.

LLMs need memory bandwidth to stream lots of data through quickly, not so much caching. Well, this is basically the same way that a GPU uses memory.

OTOH, LLM inference tends to have very predictable memory access patterns. So well-placed prefetch instructions that can execute predictable memory fetches in parallel with expensive compute might help CPU performance quite a bit. I assume that this is done already as part of optimized numerical primitives such as GEMM, since that's where most of the gain would be.

Re: RAM now represents 35 percent of bill of materials for HP PCs

#219

Post by zozbot234 »

zarzavat wrote: post #216
travisgriggs wrote: post #50

I had my formative years in programming when memory usage was something you still worried about as a programmer. And then memory expanded so much that all kinds of “optimal” patterns for programming just become nearly irrelevant. Will we start to actually consider this in software solutions again as a result?

RAM didn't get more expensive to produce. It just got more desirable. The prices will come down again when supply responds. It may take some time, but it will happen eventually.

RAM actually got more expensive to produce in the medium term because production is bottlenecked. It takes years to expand production.

Re: RAM now represents 35 percent of bill of materials for HP PCs

#220

Post by jonathanlydall »

thfuran wrote: post #93
NooneAtAll3 wrote: post #92

Earlier quoted context omitted.

most likely in a couple years this bubble will pop, just like 8 years and 16 years ago it's just a cartel cycle of gaining profits while soon eliminating all investments into competitors when flood of cheap ram "suddenly" appears

This is coming from an insane demand spike, not some nefarious plot by the RAM manufacturers.

Which is in large part due to hoarding by OpenAI.

Although their stated reason for hoarding is that they "really need it", I think it was a strategic move to make their competitors' lives more difficult with little regard for the collateral consequences to non-competitors, such as regular people or companies needing new computers.

Post reply on HN