211–220 of 536 posts
There's a future where RAM makers tool up for this massively increased demand, then the AI companies go broke as the bubble bursts, so RAM is cheap as. So laptop manufacturers get on that and start making laptops with 1TB+ memory so we can run decent LLMs on the local machine. Everyone happy :)
Earlier quoted context omitted.
> I think this will smack the hands of developers who don’t manage RAM well And hopefully kill Electron. I have never seen the point of spinning up a 300+Mb app just to display something that ought to need only 500Kb to paint onto the screen.
Who cares about 300Mb, where is that going to move the needle for you? And if the alternative is a memory-unsafe language then 300Mb is a price more than worth paying. Likewise if the alternative is the app never getting started, or being single-platform-only, because the available build systems suck too bad. There ought to be a short one-liner that anyone can run to get easily installable "binaries" for their PyQt a…
Earlier quoted context omitted.
I am betting the pendulum swings faster to the other side to excess capacity as all the construction lies of Altman fall through with financiers waking up the the fact they can't build the infrasctructure as fast nor make any profits on that infrastructure that will get built.
Those financiers can’t risk not being involved in a company with even just a slight potential for AGI.
We've been projecting both FTL and AGI as future possibilities for almost 100 years now. Do LLMs get us a lot closer to AGI? I think they get us a little closer and Moore's "law" making compute faster probably is a much bigger factor, but I think we're still a very very long ways away.
Earlier quoted context omitted.
All of the capital intensive businesses face this issue. Chemicals, Shipping, Semiconductors etc. You get market signals that the demand is there, you acquire the necessary capital, you spend 5 years to build capacity, but guess what, 5 other market players did the same thing. So now you are doomed, because the market is flooded and you have low cash flow since you need to drop prices to compete for pennies. Now you…
Forecasting demand 5 years into the future is intrinsically highly unreliable. It doesn’t matter if it is capitalism or a command economy. The bet is always going to be risky and someone will have to pay for that risk. At least with capitalism you have many different people with different perspectives on the risk making independent bets. That mitigates the more extreme negative outcomes.
if a shortage lasts years, it's not a shortage. "The market clearing price of RAM in the face of expected sustained healthy demand should lead to a stable market for years." even if gaming is and will remain very popular for years, it and the desire to upgrade gaming rigs is still a discretionary activity with more price elasticity of demand than corporate uses for RAM in the dawn of the AI age. gamers live on the ma…
It’s not merely a “gaming vs data center“. There’s so many other places DRAM and NVM are needed - mobile, automotive, other consumer electronics,… the current situation is that _all_ of that is deprived of the memory that it needs. And much of this is critical to the real economy.
letting the market set prices ensures that the chips go to the critical markets and uses. less critical uses will not allocate funds for purchases.
I’m a bit of an optimist. I think this will smack the hands of developers who don’t manage RAM well and future apps will necessarily be more memory-efficient.
The demand is being driven by inference though. I really don't think there will be much motivation.
The era of optimisation is finally here. I'm excited.
Basically, the optimizing that can happen is that I ditch heavy tools in favour of lighter ones, and hopefully enough other people do the same to help lighter tools with finances/dev resources.
Earlier quoted context omitted.
You're absolutely right? I don't really disagree with anything you're saying there, that's why I said "generally" and "in practice".
Reducing your RAM consumption is not the best approach to reducing your RAM throughput is my point. It could be effective in some specific situations, but I would definitely not say that those situations are more common than the other ones.
From what I understand, increasing cache locality is orthogonal to how much RAM an app is using. It just lets the CPU get cache hits more often, so it only relates to throughout.
That might technically offload work to the CPU, but that's work the CPU is actually good at. We want to offload that.
In the case of Electron apps, they use a lot of RAM and that's not to spare the CPU
Earlier quoted context omitted.
The demand is being driven by inference though. I really don't think there will be much motivation.
The large models are incredibly inefficient. We'll be squeezing them down for generations.
Earlier quoted context omitted.
Those financiers can’t risk not being involved in a company with even just a slight potential for AGI.
What if it takes 100 years to get to AGI or we never achieve it? All bets on AGI will just fail over and over again for decades in that case. It seems a bit like saying financiers can't risk not being involved with Faster-than-light travel technology. Yeah, it would change everything if we got it, but betting that we'll get it soon over and over again is probably not going to get you a lot of money. We've been projec…