Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ilya Sutskever: We're moving from the age of scaling to the age of research

dwarkesh.com

Post reply on HN
Print view View original

191–200 of 374 posts

Re: Ilya Sutskever: We're moving from the age of scaling to the age of research

#191

Post by AstroBen »

hiddencost wrote: post #179
pxc wrote: post #155

If "Era of Scaling" means "era of rapid and predictable performance improvements that easily attract investors", it sounds a lot like "AI summer". So... is "Era of Research" a euphemism for "AI winter"?

That presumes that performance improvements are necessary for commercialization. From what I've seen the models are smart enough, what we're lacking is the understanding and frameworks necessary to use them well. We've barely scratched the surface on commercialization. I'd argue there are two things coming: -> Era of Research -> Era of Engineering Previous AI winters happened because we didn't have a commercially via…

We still don't have a commercially viable product though?

Re: Ilya Sutskever: We're moving from the age of scaling to the age of research

#192

Post by catigula »

hiddencost wrote: post #179
pxc wrote: post #155

If "Era of Scaling" means "era of rapid and predictable performance improvements that easily attract investors", it sounds a lot like "AI summer". So... is "Era of Research" a euphemism for "AI winter"?

That presumes that performance improvements are necessary for commercialization. From what I've seen the models are smart enough, what we're lacking is the understanding and frameworks necessary to use them well. We've barely scratched the surface on commercialization. I'd argue there are two things coming: -> Era of Research -> Era of Engineering Previous AI winters happened because we didn't have a commercially via…

I don’t think the models are smart at all. I can have a speculative debate with any model about any topic and they commit egregious errors with an extremely high density.

They are, however, very good at things we’re very bad at.

Re: Ilya Sutskever: We're moving from the age of scaling to the age of research

#194

Post by echelon »

zerosizedweasle wrote: post #188
pxc wrote: post #155

If "Era of Scaling" means "era of rapid and predictable performance improvements that easily attract investors", it sounds a lot like "AI summer". So... is "Era of Research" a euphemism for "AI winter"?

If you have to ask the question, then you already know the answer

Scaling was only a meme because OpenAI kept saying all you had to do was scale the data, scale the training. The world followed.

I don't think this is the "era of research". At least not the "era of research with venture dollars" or "era of research outside of DeepMind".

I think this is the "era of applied AI" using the models we already have. We have a lot of really great stuff (particularly image and video models) that are not yet integrated into commercial workflows.

There is so much automation we can do today given the tech we just got. We don't need to invest one more dollar in training to have plenty of work to do for the next ten years.

If the models were frozen today, there are plenty of highly profitable legacy businesses that can be swapped out with AI-based solutions and workflows that are vastly superior.

For all the hoopla that image and video websites or individual foundation models get (except Nano Banana - because that's truly magical), I'm really excited about the work Adobe of all companies is doing with AI. They're the people that actually get it. The stuff they're demonstrating on their upcoming roadmap is bonkers productive and useful.

Re: Ilya Sutskever: We're moving from the age of scaling to the age of research

#195

Post by zerosizedweasle »

echelon wrote: post #194
zerosizedweasle wrote: post #188

Earlier quoted context omitted.

If you have to ask the question, then you already know the answer

Scaling was only a meme because OpenAI kept saying all you had to do was scale the data, scale the training. The world followed. I don't think this is the "era of research". At least not the "era of research with venture dollars" or "era of research outside of DeepMind". I think this is the "era of applied AI" using the models we already have. We have a lot of really great stuff (particularly image and video models)…

There's going to be a digestion period. The amount of debt, the amount of money, the number of companies that burn eye popping amounts of cash in their daily course of business. I do think there is a bright future, but after a painful period of indigestion. Too much money has been spent on the premise that scaling was all you need. A lot of money was wagered that will end up not paying off.

Re: Ilya Sutskever: We're moving from the age of scaling to the age of research

#196

Post by fzzzy »

tintor wrote: post #177
jfim wrote: post #129

Earlier quoted context omitted.

Counting letters is tricky for LLMs because they operate on tokens, not letters. From the perspective of a LLM, if you ask it "this is a sentence, count the letters in it" it doesn't see a stream of characters like we do, it sees [851, 382, 261, 21872, 11, 3605, 290, 18151, 306, 480].

So what? It knows number of letters in each token, and can sum them together.

How does it know the letters in the token?

It doesn't.

There's literally no mapping anywhere of the letters in a token.

Re: Ilya Sutskever: We're moving from the age of scaling to the age of research

#198

Post by [deleted] »

fragmede wrote: post #26
piker wrote: post #14

Earlier quoted context omitted.

Seems like he’s Lex without the Rogan association so hardcore liberal folks can listen without having to buy morality offsets. He’s good, and he’s filling a void in an established underserved genre is my take.

Tell me more about these morality offsets I can buy! I got a bunch of friends that listen to Joe Rogan, so I listen to him to know what they're talking about, but I've been doing so without these offsets, so my morality's been taking hits. Please help me before I make a human trafficking app for Andrew Tate!

[deleted]

Re: Ilya Sutskever: We're moving from the age of scaling to the age of research

#200

Post by amypetrik8 »

AstroBen wrote: post #191
hiddencost wrote: post #179

Earlier quoted context omitted.

That presumes that performance improvements are necessary for commercialization. From what I've seen the models are smart enough, what we're lacking is the understanding and frameworks necessary to use them well. We've barely scratched the surface on commercialization. I'd argue there are two things coming: -> Era of Research -> Era of Engineering Previous AI winters happened because we didn't have a commercially via…

We still don't have a commercially viable product though?

google what you just said and look at the top hit

it's a AI summary

google eats that ad revenue

it eats the whole thing

it blocked your click on the link... it drinks your milkshake

so, yes, there a 100 billion commercially viable product

Post reply on HN