Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What if AI doesn't need more RAM but better math?

adlrocha.substack.com

Post reply on HN
Print view View original

101–110 of 111 posts

Re: What if AI doesn't need more RAM but better math?

#101

Post by Dylan16807 »

throwatdem12311 wrote: post #74
lelanthran wrote: post #68

Earlier quoted context omitted.

> And when that happens people STILL won’t be able to afford the hardware. Of course they will - if that happens all these AI token providers won't have a use for all that hardware they bought. You'll be buying used H100s and H200s off eBay for pennies on the dollar.

No they won’t they’re just going to get absorbed into Azure and AWS and used for generic GPU compute that you rent until they’re burned out trash.

Then those datacenters will barely need any new GPUs, so the companies making them will be desperate to get gamers to buy cards and set very competitive prices.

Re: What if AI doesn't need more RAM but better math?

#102

Post by Dylan16807 »

Almondsetat wrote: post #31
Imustaskforhelp wrote: post #18

Earlier quoted context omitted.

> Then we can make them even bigger. But what if it becomes "good enough", that for most intents and purposes, small models can be "good enough" There are some people here/on r/localllama who I have seen run some small models and sometimes even run multiple of them to solve/iterate quickly and have a larger model plug into it and fix anything remaining. This would still mean that larger/SOTA models might have some de…

Because the true goal is AGI, not just nice little tools to solve subsets of problems. The first company which can achieve human level intelligence will just be able to self-improve at such a rate as to create a gigantic moat

There's no particular reason to assume a human level AI would be able to improve itself any better than the thousands of human level humans that designed it.

Re: What if AI doesn't need more RAM but better math?

#103

Post by Almondsetat »

Dylan16807 wrote: post #102
Almondsetat wrote: post #31

Earlier quoted context omitted.

Because the true goal is AGI, not just nice little tools to solve subsets of problems. The first company which can achieve human level intelligence will just be able to self-improve at such a rate as to create a gigantic moat

There's no particular reason to assume a human level AI would be able to improve itself any better than the thousands of human level humans that designed it.

Sure, but: that single human with the intelligence of a top tier engineer of scientist will have immediate access to all human knowledge. Plus, what do you think happens the moment its optimizes itself to run in 2, 4, 8, 16, etc. parallel instances?

Re: What if AI doesn't need more RAM but better math?

#104

Post by iLemming »

high_na_euv wrote: post #100
iLemming wrote: post #93

Earlier quoted context omitted.

Raising, nurturing, training, and mentoring an expert mathematician is not cheap; it never was, perhaps the first time in history when we can witness that rule to change - spinning up a bunch of math-savvy agents, each smarter than Ramanujan maybe will get too cheap.

You dont have to raise them, someone already did it, you have to hire them

You're oversimplifying the message I'm trying to convey. "you just hire them, someone already raised them" - treats mathematicians as a commodity stock rather than a flow. The conversation frames it as "mathematicians vs. RAM" - a cost comparison. But that's like comparing the cost of a GPS unit vs. a ship captain. The captain isn't expensive because they can calculate routes; they're expensive because they know when the route is wrong. AI makes the math cheaper but makes the mathematician more valuable, at least until true AGI genuinely surpasses human mathematical creativity - at which point we have much bigger economic questions than mathematician salaries.

The topic on itself is quite interesting, and far complex than supply/demand norms. Even before AI, there was and both wasn't shortage of mathematicians - academic pure mathematics - there's a glut. High school teachers - people exist; but they won't work for teacher salaries. Applied math - acute shortage - quant finance, ML research, cryptography, pharmaceutical modeling - we don't have enough. NSA - always struggled to hire - private sector salaries pull people away. Interdisciplinary - mathematical biology, climate modeling, materials science - domains where math is the bottleneck but the job title isn't really "mathematician" - acute shortage.

Re: What if AI doesn't need more RAM but better math?

#105

Post by Dylan16807 »

Almondsetat wrote: post #103
Dylan16807 wrote: post #102

Earlier quoted context omitted.

There's no particular reason to assume a human level AI would be able to improve itself any better than the thousands of human level humans that designed it.

Sure, but: that single human with the intelligence of a top tier engineer of scientist will have immediate access to all human knowledge. Plus, what do you think happens the moment its optimizes itself to run in 2, 4, 8, 16, etc. parallel instances?

Well, A) "top tier engineer/scientist" is a significant step above generic human, B) the human engineers/scientists also have immediate access to the same database, C) The humans have been optimizing it for even longer, so what makes us think the AI can optimize itself even a couple percent?

For example, if the number of AIs you can run per petaflop started to scale with the cube root of researcher-years, then even if your researcher AIs are quite fast and you can double your density in a couple years, hitting 5x will take a decade and hitting 10x will approach half a century.

Re: What if AI doesn't need more RAM but better math?

#106

Post by LtWorf »

Yokohiii wrote: post #30
abdelhousni wrote: post #10

The same could be said about other IT domain... When you see single webpages that weight by tens of MB you wonder how we came to this.

Detachment from reality. Code elegance is more important then anything else. As simple as that.

You've never seen the sources for such pages I presume?

Re: What if AI doesn't need more RAM but better math?

#107

Post by SphericalCowww »

fittingopposite wrote: post #98
SphericalCowww wrote: post #59

I mean, since GPT-4, I believe the RAM is no longer creating the miracle that the LLM performance scales directly with the model size. At least ChatGPT itself convinced me that any decent-sized company can create a GPT4 equivalent in terms of model size, but limited by service options, like memory cache and hallucination handling. Companies buy RAM simply to ride the stock hype. I am no expert, so this is a shallow t…

Re continuous fine-tuning: how do you avoid catastrophic forgetting in your proposal?

My understanding is that this is what the LoRAs are for; my belief is that they serve as "memory" to their live observations (a more NN-like cache, say), while the main LLM remains unchanged. These LoRAs are also weighted, so that LoRAs irrelevant to the current task will not be trained, while the relevant LoRAs will be reinforced.

But I never built it, so I am not sure if such an emergent state will appear or not.

Re: What if AI doesn't need more RAM but better math?

#108

Post by nkmnz »

cyanydeez wrote: post #50
nkmnz wrote: post #19

Earlier quoted context omitted.

Citation needed. I've heard this quite often, but so far, I haven't seen proof of the stated causality. PS: This doesn't mean that better public transportation could deliver more bang for the buck than the n-th additional car lane. But never ever have I heard from anybody that they chose to buy a car or use an existing car more often because an additional lane has been built.

You've never heard anyone choose to take side streets instead of the highway because of traffic jams? No one ever goes out of their way to avoid heavily trafficed areas?

I don't understand what the point is you're trying to make. When people at t0 take detours because of traffic jams on the direct route, and then at t1, there are less traffic jam on the direct route due to additional lanes, so they decide to take the direct route, then total traffic is down, because they no longer take a detour. Even if they are still part of a newly induced traffic jam.

Re: What if AI doesn't need more RAM but better math?

#109

Post by jurgenburgen »

Almondsetat wrote: post #31
Imustaskforhelp wrote: post #18

Earlier quoted context omitted.

> Then we can make them even bigger. But what if it becomes "good enough", that for most intents and purposes, small models can be "good enough" There are some people here/on r/localllama who I have seen run some small models and sometimes even run multiple of them to solve/iterate quickly and have a larger model plug into it and fix anything remaining. This would still mean that larger/SOTA models might have some de…

Because the true goal is AGI, not just nice little tools to solve subsets of problems. The first company which can achieve human level intelligence will just be able to self-improve at such a rate as to create a gigantic moat

There’s no evidence that the current architectures will reach AGI levels.

Of course OpenAI wants you to think they will rule the world but if we’ve reached the plateau of LLM capabilities regardless of the amount of compute we throw at them then local models will soon be good enough.

Re: What if AI doesn't need more RAM but better math?

#110

Post by SphericalCowww »

redanddead wrote: post #77
SphericalCowww wrote: post #59

I mean, since GPT-4, I believe the RAM is no longer creating the miracle that the LLM performance scales directly with the model size. At least ChatGPT itself convinced me that any decent-sized company can create a GPT4 equivalent in terms of model size, but limited by service options, like memory cache and hallucination handling. Companies buy RAM simply to ride the stock hype. I am no expert, so this is a shallow t…

share it on gh and make a show hn post about it, maybe you're right the models are still very stupid atm something needs to change

I put all the conceptual ideas here (spiced with far-fetched claims): https://github.com/SphericalCowww/ML_LunaLoRA

But I think my hn level is too low, would really like some expert's opinion, though, whelp... Meanwhile, let me implement some basic models first.

Post reply on HN