Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The RAM shortage could last years

theverge.com

Post reply on HN
Print view View original

291–300 of 536 posts

Re: The RAM shortage could last years

#291

Post by paganel »

Epa095 wrote: post #283
drnick1 wrote: post #199

Earlier quoted context omitted.

This just highlights what an utter failure and self-inflicted wound the green policies of Euro countries have been. Europe has already lost the AI race to the U.S. and China.

Renewables is the only realistic path to energy independence. Today's global situation should show the absolute necessity of that, even if you dont give a sh*t about the environment. Had we done more 10 years ago we would have been better of. The second best time to start is now.

Europe still has coal, lots of it, not using it is a political choice and a self-inflicted wound.

Re: The RAM shortage could last years

#292

Post by mayama »

bluGill wrote: post #193
formerly_proven wrote: post #102

Earlier quoted context omitted.

DRAM and to a lesser degree storage are notorious for their feast and famine cycles (Well that and collusion)

There are a lot of cyclical businesses that make money every year. It requires careful management. Factories can produce less than full capacity - but you better design for that. you can make money in the worst years without laying anyone off even - but it requires careful attention to details and not over hiring in good times as if they will never end.

Last year ai folks are all over wallstreet and articles, decrying how hardware folks are a roadblock to new frontiers in AI. They just couldn't print and pack the new chips faster.

Re: The RAM shortage could last years

#293

Post by zozbot234 »

locknitpicker wrote: post #256
LastTrain wrote: post #247

Something I haven’t been able to reconcile: If AI makes software easier to create, that will drive the price down. How are software companies going to make enough revenue to pay for AI, when the amount of money being spent on AI is already multiples of the current total global expenditure on software? This demand for RAM is built on a foundation of sand, there will be a glut of capacity when it all shakes out.

> If AI makes software easier to create, that will drive the price down. Supposedly AI drives down the cost of producing software,not the "price". > How are software companies going to make enough revenue to pay for AI, when the amount of money being spent on AI is already multiples of the current total global expenditure on software? Currently, the cost of AI is between $20/month and around $200/month per developer.…

> Who knows? What I know is that I need >64GB of RAM to run local models, and that means most people will need to upgrade from their 8Gb/16GB setup to do the same. Graphics cards follow mostly the same pattern.

Depends how big the models are, how fast you want them to run and how much context you need for your usage. If you're okay with running only smaller models (which are still very capable in general, their main limitation is world knowledge) making very simple inferences at low overall throughput, you can just repurpose the RAM, CPUs/iGPUs and storage in the average setup.

Re: The RAM shortage could last years

#294

Post by cestith »

buccal wrote: post #258
raydev wrote: post #230

Earlier quoted context omitted.

It won't be "happening" until Slack, Teams, and Discord leave Electron behind. They are the apps that need to be open 24/7.

Teams works similarly in browser tab and "natively". Slack was similar if I remember correctly.

You should check the memory use of that browser tab. You’re not saving much either way running in a browser or in Electron, which is effectively a browser.

Re: The RAM shortage could last years

#295

Post by thelastgallon »

It will last forever. After covid, all manufacturers understood the value of limiting supply and extracting profits. Cars used to super cheap before covid, they will never go back to the same levels.

From now on, RAM will always be super costly for consumers, because they can't make massive deals like Apple/OpenAI/etc. We are the bagholders.

Re: The RAM shortage could last years

#296

Post by anhner »

brikym wrote: post #225
rekabis wrote: post #121

Earlier quoted context omitted.

> I think this will smack the hands of developers who don’t manage RAM well And hopefully kill Electron. I have never seen the point of spinning up a 300+Mb app just to display something that ought to need only 500Kb to paint onto the screen.

It's happening. Cursor 3 moved to rust. A lot of people are using Zed (rust) instead of vscode.

"cursor 3" is just a landing page. The editor is still the old vscode fork...

Re: The RAM shortage could last years

#297

Post by nirui »

LastTrain wrote: post #247

Something I haven’t been able to reconcile: If AI makes software easier to create, that will drive the price down. How are software companies going to make enough revenue to pay for AI, when the amount of money being spent on AI is already multiples of the current total global expenditure on software? This demand for RAM is built on a foundation of sand, there will be a glut of capacity when it all shakes out.

> This demand for RAM is built on a foundation of sand

Not exactly.

LLMs are already quite useful today if you use them as a tool, so they are there to stay. The remaining problem is scalability, a.k.a. how to make LLMs cheap to use.

But scalability is not really a requirement when you look the bigger picture. If smaller software company/projects can't afford to use AI, the bigger ones might just. Eventually they will discover variable use cases for such tech, even if it only serves big firms i.e. defense, resource extraction, war, finance etc.

To the other end, if scalability is achieved, the use of LLM products will be cheaper too, so smaller project can also use them. But of course, if LLM usage is too cheap, then many were-to-be-consumers will just create software projects by themselves at their homes.

Re: The RAM shortage could last years

#298

Post by Ray20 »

raydev wrote: post #230
brikym wrote: post #225

Earlier quoted context omitted.

It's happening. Cursor 3 moved to rust. A lot of people are using Zed (rust) instead of vscode.

It won't be "happening" until Slack, Teams, and Discord leave Electron behind. They are the apps that need to be open 24/7.

It's not entirely clear what the connection is.

We're not doing Electron because some popular software also using it. We're doing Electron because the ability to create truly cross-platform interfaces with the web stack is more important to us than 300 MB of user memory.

Re: The RAM shortage could last years

#299

Post by parl_match »

torginus wrote: post #42
stuxnet79 wrote: post #11

Ok so Samsung, SK Hynix and Micron do not have the capacity to meet demand. Also, what little capacity they do have they are allocating to HBM over DRAM. Based on my limited knowledge HBM can not be easily repurposed for consumer electronics. Translation: main street is cooked for the next 3-4 years. It doesn't stop there though. OpenAI is currently mired in a capital crunch. Their last round just about sucked all th…

The Radeon VII came out in 2019 as a $700 consumer GPU with an 1TB/s HBM2 memory subsystem which is more than any consumer GPU you can get today, including the high-end ones afaik. At that point in time, there was a whole lineup of AMD GPUs with HBM going down into the midrange. If they could make this stuff and sell it to regular people a decade ago for very palatable prices, why do they come up with the idea that t…

> why do they come up with the idea that this is the technology of the gods, unaffordable by mere mortals?

because the gods want it all and are willing to pay top dollar.

Re: The RAM shortage could last years

#300

Post by adrianN »

AnthonyMouse wrote: post #103
estimator7292 wrote: post #82

Earlier quoted context omitted.

Is that relevant? The grid in every country is getting ridiculously stressed by datacenters.

What the grid looks like in different countries is very different. The Dutch power grid is already almost 50% renewables, which is an inconvenience for adding capacity because that's around where you have to start really dealing with storage in order to add more. In most other places the percentage is significantly less than that and then you can easily add more of the cheap-but-intermittent stuff because a cloudy da…

I don’t think the source of the electricity is particularly relevant to whether or not you have the transport capacity to add tens of megawatts of demand to the grid. The problem is generally not the supply but whether your local transformers have capacity left.
Post reply on HN