Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What if AI doesn't need more RAM but better math?

adlrocha.substack.com

Post reply on HN
Print view View original

41–50 of 111 posts

Re: What if AI doesn't need more RAM but better math?

#41

Post by 9rx »

Almondsetat wrote: post #31
Imustaskforhelp wrote: post #18

Earlier quoted context omitted.

> Then we can make them even bigger. But what if it becomes "good enough", that for most intents and purposes, small models can be "good enough" There are some people here/on r/localllama who I have seen run some small models and sometimes even run multiple of them to solve/iterate quickly and have a larger model plug into it and fix anything remaining. This would still mean that larger/SOTA models might have some de…

Because the true goal is AGI, not just nice little tools to solve subsets of problems. The first company which can achieve human level intelligence will just be able to self-improve at such a rate as to create a gigantic moat

> The first company which can achieve human level intelligence will just be able to...

They say prostitution is the oldest industry of all. We know how to achieve human-level intelligence quite well. The outstanding challenge is figuring out how to produce an energy efficient human-level intelligence.

Re: What if AI doesn't need more RAM but better math?

#43

Post by Eddy_Viscosity2 »

Verdex wrote: post #39
fph wrote: post #7

Despite the shortage, RAM is still cheaper than mathematicians.

It's also less frustrating to organize world wide ram production and logistics than to deal with a single mathematician. Constantly sitting around trying to solve problems that nobody has made headway on for hundreds of years. Or inventing theorems around 15th century mysticism that won't be applicable for hundreds of years. Now if you'll excuse me I need to multiply some numbers by 3 and divide them by 2 ... I'm so…

The comment feels a bit like Verdex may have dated a mathematician at some point and it went sour.

Re: What if AI doesn't need more RAM but better math?

#44

Post by delecti »

jurgenburgen wrote: post #4
LoganDark wrote: post #3

We will not see memory demand decrease because this will simply allow AI companies to run more instances. They still want an infinite amount of memory at the moment, no matter how AI improves.

If models become more efficient we will move more of the work to local devices instead of using SaaS models. We’re still in the mainframe era of LLM.

As I understand this advancement, this doesn't let you run bigger models, it lets you maintain more chat context. So Anthropic and OpenAI won't need as much hardware running inference to serve their users, but it doesn't do much to make bigger models work on smaller hardware.

Though I'm not an expert, maybe my understanding of the memory allocation is wrong.

Re: What if AI doesn't need more RAM but better math?

#45

Post by simne »

Sure, we need better math, it is obvious.

Unfortunately, nobody at big companies know, what exactly math will win, so competition not end.

So, researchers will try one solution, then other solution, etc, until find something perfect, or until semiconductors production (Moore's Law) made enough semiconductors to run current models fast enough.

I believe, somebody already have silver bullet of ideal AI algorithm, which will lead all us to AGI, when scaled in some big company, but this knowledge is not obvious at the moment.

Re: What if AI doesn't need more RAM but better math?

#47

Post by vidarh »

singpolyma3 wrote: post #40
mustyoshi wrote: post #34

Earlier quoted context omitted.

I don't see how we'll ever get to widespread local LLM. The power efficiency alone is a strong enough pressure to use centralized model providers. My 3090 running 24b or 32b models is fun, but I know I'm paying way more per token in electricity, on top of lower quality tokens. It's fun to run them locally, but for anything actually useful it's cheaper to just pay API prices currently.

Until you put up your solar and then power is almost free...

The amortised cost including the panels and labour is nowhere near "almost free".

Re: What if AI doesn't need more RAM but better math?

#50

Post by cyanydeez »

nkmnz wrote: post #19
cyanydeez wrote: post #16

Earlier quoted context omitted.

Department of Transportation always thinks adding more lanes will reduce traffic. It doesn't, it induces demand. Why? Because there's always too many people with cars who will fill those lanes.

Citation needed. I've heard this quite often, but so far, I haven't seen proof of the stated causality. PS: This doesn't mean that better public transportation could deliver more bang for the buck than the n-th additional car lane. But never ever have I heard from anybody that they chose to buy a car or use an existing car more often because an additional lane has been built.

You've never heard anyone choose to take side streets instead of the highway because of traffic jams? No one ever goes out of their way to avoid heavily trafficed areas?
Post reply on HN