Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What if AI doesn't need more RAM but better math?

adlrocha.substack.com

Post reply on HN
Print view View original

31–40 of 111 posts

Re: What if AI doesn't need more RAM but better math?

#31

Post by Almondsetat »

Imustaskforhelp wrote: post #18
Ray20 wrote: post #17

Earlier quoted context omitted.

> If models become more efficient Then we can make them even bigger.

> Then we can make them even bigger. But what if it becomes "good enough", that for most intents and purposes, small models can be "good enough" There are some people here/on r/localllama who I have seen run some small models and sometimes even run multiple of them to solve/iterate quickly and have a larger model plug into it and fix anything remaining. This would still mean that larger/SOTA models might have some de…

Because the true goal is AGI, not just nice little tools to solve subsets of problems. The first company which can achieve human level intelligence will just be able to self-improve at such a rate as to create a gigantic moat

Re: What if AI doesn't need more RAM but better math?

#33

Post by Yokohiii »

> If I were Google, I wouldn’t release research that exposes a competitive advantage.

Isn't that a classic tit for tat decision and head for a loss?

Excellence and prestige are valuable too. You get those expensive ML for a small discount, public/professional perception, etc. Considering the public communication from Google, that isn't complete sociopathic, they know this war isn't won in one night, they are the only sustainably funded company in the competition. Surely they are at risk with their business, but can either go rampant or focus. They decided to focus.

Re: What if AI doesn't need more RAM but better math?

#34

Post by mustyoshi »

jurgenburgen wrote: post #4
LoganDark wrote: post #3

We will not see memory demand decrease because this will simply allow AI companies to run more instances. They still want an infinite amount of memory at the moment, no matter how AI improves.

If models become more efficient we will move more of the work to local devices instead of using SaaS models. We’re still in the mainframe era of LLM.

I don't see how we'll ever get to widespread local LLM.

The power efficiency alone is a strong enough pressure to use centralized model providers.

My 3090 running 24b or 32b models is fun, but I know I'm paying way more per token in electricity, on top of lower quality tokens.

It's fun to run them locally, but for anything actually useful it's cheaper to just pay API prices currently.

Re: What if AI doesn't need more RAM but better math?

#35

Post by naasking »

throwatdem12311 wrote: post #13
jurgenburgen wrote: post #4

Earlier quoted context omitted.

If models become more efficient we will move more of the work to local devices instead of using SaaS models. We’re still in the mainframe era of LLM.

The hyperscalers do not want us running models at the edge and they will spend infinite amounts of circular fake money to ensure hardware remains prohibitively expensive forever.

> and they will spend infinite amounts of circular fake money to ensure hardware remains prohibitively expensive forever.

That's ridiculous, "infinite money" isn't a thing. They will spend as much as they can not because they want to keep local solutions out, but because it enables them to provide cheaper services and capture more of the market. We all eventually benefit from that.

Re: What if AI doesn't need more RAM but better math?

#36

Post by Ray20 »

Imustaskforhelp wrote: post #18
Ray20 wrote: post #17

Earlier quoted context omitted.

> If models become more efficient Then we can make them even bigger.

> Then we can make them even bigger. But what if it becomes "good enough", that for most intents and purposes, small models can be "good enough" There are some people here/on r/localllama who I have seen run some small models and sometimes even run multiple of them to solve/iterate quickly and have a larger model plug into it and fix anything remaining. This would still mean that larger/SOTA models might have some de…

> But what if it becomes "good enough", that for most intents and purposes, small models can be "good enough"

It's simple: then we'll make our intents and purposes bigger.

Re: What if AI doesn't need more RAM but better math?

#37

Post by Verdex »

konaraddi wrote: post #26

> applying this compression algorithm at scale may significantly relax the memory bottleneck issue. I don’t think they’re going to downsize though, I think the big players are just going to use the freed up memory for more workflows or larger models because the big players want to scale up. It’s a cat and mouse race for the best models.

Known in the business as 'pulling a jevons'

Re: What if AI doesn't need more RAM but better math?

#39

Post by Verdex »

fph wrote: post #7

Despite the shortage, RAM is still cheaper than mathematicians.

It's also less frustrating to organize world wide ram production and logistics than to deal with a single mathematician.

Constantly sitting around trying to solve problems that nobody has made headway on for hundreds of years. Or inventing theorems around 15th century mysticism that won't be applicable for hundreds of years.

Now if you'll excuse me I need to multiply some numbers by 3 and divide them by 2 ... I'm so close guys.

Re: What if AI doesn't need more RAM but better math?

#40

Post by singpolyma3 »

mustyoshi wrote: post #34
jurgenburgen wrote: post #4

Earlier quoted context omitted.

If models become more efficient we will move more of the work to local devices instead of using SaaS models. We’re still in the mainframe era of LLM.

I don't see how we'll ever get to widespread local LLM. The power efficiency alone is a strong enough pressure to use centralized model providers. My 3090 running 24b or 32b models is fun, but I know I'm paying way more per token in electricity, on top of lower quality tokens. It's fun to run them locally, but for anything actually useful it's cheaper to just pay API prices currently.

Until you put up your solar and then power is almost free...
Post reply on HN