Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ilya Sutskever: We're moving from the age of scaling to the age of research

dwarkesh.com

Post reply on HN
Print view View original

231–240 of 374 posts

Re: Ilya Sutskever: We're moving from the age of scaling to the age of research

#231

Post by coder-3 »

joelthelion wrote: post #227

> When do you expect that impact? I think the models seem smarter than their economic impact would imply. > Yeah. This is one of the very confusing things about the models right now. As someone who's been integrating "AI" and algorithms into people's workflows for twenty years, the answer is actually simple. It takes time to figure out how exactly to use these tools, and integrate them into existing tooling and workf…

As someone who is building an LLM-powered product on the side, using AI coding agents to help with development of said LLM-powered product and for my day job, and has a long-tail of miscellaneous uses for AI, I suspect you're right.

Re: Ilya Sutskever: We're moving from the age of scaling to the age of research

#232

Post by giardini »

andy_ppp wrote: post #2

So is the translation endless scaling has stopped being as effective?

I'll be convinced LLMs are a reasonable approach to AI when an LLM can give reasonable answers after being trained with approximately the same books and classes in school that I was once I completed my college education.

Re: Ilya Sutskever: We're moving from the age of scaling to the age of research

#233

Post by giardini »

4b11b4 wrote: post #223
nsoonhui wrote: post #219

One thing I’m curious about is this: Ilya Sutskever wants to build Safe Superintelligence, but he keeps his company and research very secretive. Given that building Safe Superintelligence is extraordinarily difficult — and no single person’s ideas or talents could ever be enough — how does secrecy serve that goal?

Doesn't sound like you listened to the interview. He addresses this and says he may make releases that would be otherwise held back because he believes it's important for developments to be seen by the public.

No reasonable person would do that! That is, if you had the key to AI, you wouldn't share it and you would do everything possible to prevent it's dissemination. Meanwhile you would use it to conquer the world! Bwahahahaaaah!

Re: Ilya Sutskever: We're moving from the age of scaling to the age of research

#234

Post by simianparrot »

selectodude wrote: post #210
ares623 wrote: post #185

Earlier quoted context omitted.

The labs can't just stop improvements though. They made promises. And the capacity to run the current models are subsidized by those promises. If the promise is broken, then the capacity goes with it.

> the capacity goes with it. Sort of. The GPUs exist. Maybe LLM subs can’t pay for electricity plus $50,000 GPUs, but I bet after some people get wiped out, there’s a market there.

Datacenter GPU's have a lifespan of 1-3 years depending on use. So yes they exist, but not for long, unless they go entirely unused. But then they also deprecate in efficiency compared to new hardware extremely fast as well, so their shelf life is severely limited either way.

Re: Ilya Sutskever: We're moving from the age of scaling to the age of research

#235

Post by otabdeveloper4 »

joelthelion wrote: post #227

> When do you expect that impact? I think the models seem smarter than their economic impact would imply. > Yeah. This is one of the very confusing things about the models right now. As someone who's been integrating "AI" and algorithms into people's workflows for twenty years, the answer is actually simple. It takes time to figure out how exactly to use these tools, and integrate them into existing tooling and workf…

> the models seem smarter than their economic impact would imply

Key word is "seem".

Re: Ilya Sutskever: We're moving from the age of scaling to the age of research

#236

Post by soulofmischief »

simianparrot wrote: post #234
selectodude wrote: post #210

Earlier quoted context omitted.

> the capacity goes with it. Sort of. The GPUs exist. Maybe LLM subs can’t pay for electricity plus $50,000 GPUs, but I bet after some people get wiped out, there’s a market there.

Datacenter GPU's have a lifespan of 1-3 years depending on use. So yes they exist, but not for long, unless they go entirely unused. But then they also deprecate in efficiency compared to new hardware extremely fast as well, so their shelf life is severely limited either way.

At this pace, it won't be many years before the industry is dependent on resource wars in order to sustain itself.

Re: Ilya Sutskever: We're moving from the age of scaling to the age of research

#237

Post by otabdeveloper4 »

casey2 wrote: post #212
pxc wrote: post #155

If "Era of Scaling" means "era of rapid and predictable performance improvements that easily attract investors", it sounds a lot like "AI summer". So... is "Era of Research" a euphemism for "AI winter"?

Not quite, there are still trillions of dollars to burn through. We'll probably get some hardware that can accelerate LLM training and inference a million times, but still won't even be close to AGI It's interesting to think about what emotions/desires an AI would need to improve

The actual business model is in local, offline commodity consumer LLM devices. (Think something the size and cost of a wi-fi router.)

This won't happen until Chinese manufacturers get the manufacturing capacity to make these for cheap.

I.e., not in this bubble and you'll have to wait a decade or more.

Re: Ilya Sutskever: We're moving from the age of scaling to the age of research

#238

Post by nsomaru »

simianparrot wrote: post #234
selectodude wrote: post #210

Earlier quoted context omitted.

> the capacity goes with it. Sort of. The GPUs exist. Maybe LLM subs can’t pay for electricity plus $50,000 GPUs, but I bet after some people get wiped out, there’s a market there.

Datacenter GPU's have a lifespan of 1-3 years depending on use. So yes they exist, but not for long, unless they go entirely unused. But then they also deprecate in efficiency compared to new hardware extremely fast as well, so their shelf life is severely limited either way.

Personally I am waiting for the day I can realistically buy a second hand three year old datacentre GPU so I can run Kimi K2 in my shed. Given enough time, not a pipe dream. But 10 years at least.

Re: Ilya Sutskever: We're moving from the age of scaling to the age of research

#240

Post by jiriknesl »

joelthelion wrote: post #227

> When do you expect that impact? I think the models seem smarter than their economic impact would imply. > Yeah. This is one of the very confusing things about the models right now. As someone who's been integrating "AI" and algorithms into people's workflows for twenty years, the answer is actually simple. It takes time to figure out how exactly to use these tools, and integrate them into existing tooling and workf…

Oh yes, this is 100% accurate.

Very often, when designing ERP, or other system, people think: "This is easy, I just this XYZ I am done." Then, you find that there are many corner use-cases. XYZ can be split to phases, you might need to add approvals, logging, data integrations... and what was a simple task, becomes 10 tasks.

In the first year of CompSci uni, our teacher told us a thing I remember: Every system is 90% finished 90% of time. He was right.

Post reply on HN