Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What if AI doesn't need more RAM but better math?

adlrocha.substack.com

Post reply on HN
Print view View original

21–30 of 111 posts

Re: What if AI doesn't need more RAM but better math?

#21

Post by topspin »

throwatdem12311 wrote: post #13
jurgenburgen wrote: post #4

Earlier quoted context omitted.

If models become more efficient we will move more of the work to local devices instead of using SaaS models. We’re still in the mainframe era of LLM.

The hyperscalers do not want us running models at the edge and they will spend infinite amounts of circular fake money to ensure hardware remains prohibitively expensive forever.

> they will spend infinite amounts of circular fake money > forever

If that's the plan (there is no plan) then it expires at some point, because it's a spiral and such spirals always bottom out.

Re: What if AI doesn't need more RAM but better math?

#23

Post by j16sdiz »

nkmnz wrote: post #19
cyanydeez wrote: post #16

Earlier quoted context omitted.

Department of Transportation always thinks adding more lanes will reduce traffic. It doesn't, it induces demand. Why? Because there's always too many people with cars who will fill those lanes.

Citation needed. I've heard this quite often, but so far, I haven't seen proof of the stated causality. PS: This doesn't mean that better public transportation could deliver more bang for the buck than the n-th additional car lane. But never ever have I heard from anybody that they chose to buy a car or use an existing car more often because an additional lane has been built.

Have you tried the "Reference" section on the Wikipedia article?

https://en.wikipedia.org/wiki/Induced_demand#cite_note-vande...

Re: What if AI doesn't need more RAM but better math?

#24

Post by throwatdem12311 »

topspin wrote: post #21
throwatdem12311 wrote: post #13

Earlier quoted context omitted.

The hyperscalers do not want us running models at the edge and they will spend infinite amounts of circular fake money to ensure hardware remains prohibitively expensive forever.

> they will spend infinite amounts of circular fake money > forever If that's the plan (there is no plan) then it expires at some point, because it's a spiral and such spirals always bottom out.

And when that happens people STILL won’t be able to afford the hardware.

Re: What if AI doesn't need more RAM but better math?

#25

Post by staminade »

tornikeo wrote: post #14

Sigh. Don't make me tap the sign [1] [1] http://www.incompleteideas.net/IncIdeas/BitterLesson.html

Doesn't seem relevant here. TurboQuant isn't a domain-specific technique like the BL is talking about, it's a general optimisation for transformers that helps leverage computation more effectively.

Re: What if AI doesn't need more RAM but better math?

#26

Post by konaraddi »

> applying this compression algorithm at scale may significantly relax the memory bottleneck issue.

I don’t think they’re going to downsize though, I think the big players are just going to use the freed up memory for more workflows or larger models because the big players want to scale up. It’s a cat and mouse race for the best models.

Post reply on HN