Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ilya Sutskever: We're moving from the age of scaling to the age of research

dwarkesh.com

Post reply on HN
Print view View original

101–110 of 374 posts

Re: Ilya Sutskever: We're moving from the age of scaling to the age of research

#101

Post by [deleted] »

Herring wrote: post #66
rockinghigh wrote: post #49

Earlier quoted context omitted.

You should read the transcript. He's including 2025 in the age of scaling. > Maybe here’s another way to put it. Up until 2020, from 2012 to 2020, it was the age of research. Now, from 2020 to 2025, it was the age of scaling—maybe plus or minus, let’s add error bars to those years—because people say, “This is amazing. You’ve got to scale more. Keep scaling.” The one word: scaling. > But now the scale is so big. Is th…

Nope, Epoch.ai thinks we have enough to scale till 2030 at least. https://epoch.ai/blog/can-ai-scaling-continue-through-2030 ^ /_\ ***

[deleted]

Re: Ilya Sutskever: We're moving from the age of scaling to the age of research

#102

Post by baxtr »

mrandish wrote: post #72
baxtr wrote: post #41

Earlier quoted context omitted.

Being well known doesn’t qualify as a moat.

Agreed. But it can be a significant growth boost. Senior partners at high-profile VCs will meet with them. Early key hires they are trying to recruit will be favorably influenced by their reputation. The media will probably cover whatever they launch, accelerating early user adoption. Of course, the product still has to generate meaningful value - but all these 'buffs' do make several early startup challenges signifi…

Yes, I am not debating that it gets you a significant boost.

I’m personally not aware of a strong correlation with real business value created after the initial boost phase. But surely there must be examples.

Re: Ilya Sutskever: We're moving from the age of scaling to the age of research

#103

Post by rdedev »

Herring wrote: post #30

https://metr.org/blog/2025-03-19-measuring-ai-ability-to-com... He’s wrong we still scaling, boys.

The 3rd graph is interesting. Once the model performance reaches above human baseline, the growth seems to be logarithmic instead of exponential.

Re: Ilya Sutskever: We're moving from the age of scaling to the age of research

#104

Post by imiric »

Herring wrote: post #66
rockinghigh wrote: post #49

Earlier quoted context omitted.

You should read the transcript. He's including 2025 in the age of scaling. > Maybe here’s another way to put it. Up until 2020, from 2012 to 2020, it was the age of research. Now, from 2020 to 2025, it was the age of scaling—maybe plus or minus, let’s add error bars to those years—because people say, “This is amazing. You’ve got to scale more. Keep scaling.” The one word: scaling. > But now the scale is so big. Is th…

Nope, Epoch.ai thinks we have enough to scale till 2030 at least. https://epoch.ai/blog/can-ai-scaling-continue-through-2030 ^ /_\ ***

That article is from August 2024. A lot has changed since then.

Specifically, performance of SOTA models has been reaching a plateau on all popular benchmarks, and this has been especially evident in 2025. This is why every major model announcement shows comparisons relative to other models, but not a historical graph of performance over time. Regardless, benchmarks are far from being a reliable measurement of the capabilities of these tools, and they will continue to be reinvented and gamed, but the asymptote is showing even on their own benchmarks.

We can certainly continue to throw more compute at the problem. But the point is that scaling the current generation of tech will continue to have fewer returns.

To make up for this, "AI" companies are now focusing on engineering. 2025 has been the year of MCP, "agents", "skills", etc., which will continue in 2026. This is a good thing, as these tools need better engineering around them, so they can deliver actual value. But the hype train is running out of steam, and unless there is a significant breakthrough soon, I suspect that next year will be a turning point in this hype cycle.

Re: Ilya Sutskever: We're moving from the age of scaling to the age of research

#105

Post by [deleted] »

l5870uoo9y wrote: post #78

> These models somehow just generalize dramatically worse than people. The whole mess surrounding Grok's ridiculous overestimation of Elon's abilities in comparison to other world stars, did not so much show Grok's sycophancy or bias towards Elon, as it showed that Grok fundamentally cannot compare (generalize) or has a deeper understanding of what the generated text is about. Calling for more research and less scali…

[deleted]

Re: Ilya Sutskever: We're moving from the age of scaling to the age of research

#106

Post by JimmyBuckets »

I respect Ilya hugely as a researcher in ML and quite admire his overall humility, but I have to say I cringed quite a bit at the start of this interview when he talks about emotions, their relative complexity, and origin. Emotion is so complex, even taking all the systems in the body that it interacts with. And many mammals have very intricate socio-emotional lives - take Orcas or Elephants. There is an arrogance I have seen that is typical of ML (having worked in the field) that makes its members too comfortable trodding into adjacent intellectual fields they should have more respect and reverence for. Anyone else notice this? It's something physicists are often accused of also.

Re: Ilya Sutskever: We're moving from the age of scaling to the age of research

#107

Post by signatoremo »

l5870uoo9y wrote: post #93
radicaldreamer wrote: post #84

Earlier quoted context omitted.

I think the problem with that is that Grok has likely been prompted to do that in the system prompt or some prompts that get added for questions about Elon. That doesn't reflect on the actual reasoning or generalization abilities of the underlying model most likely.

You can also give AI models Nobel-prize winning world literature and ask why this is bad and they will tear apart the text, without ever thinking "wait this is some of the best writing produced by man".

I bet that you can find plenty of exactly that from the human reviews of any past winner.

Re: Ilya Sutskever: We're moving from the age of scaling to the age of research

#108

Post by ffsm8 »

l5870uoo9y wrote: post #93
radicaldreamer wrote: post #84

Earlier quoted context omitted.

I think the problem with that is that Grok has likely been prompted to do that in the system prompt or some prompts that get added for questions about Elon. That doesn't reflect on the actual reasoning or generalization abilities of the underlying model most likely.

You can also give AI models Nobel-prize winning world literature and ask why this is bad and they will tear apart the text, without ever thinking "wait this is some of the best writing produced by man".

At least Claude will absolutely tell you if it determines something is on point, even if you explicitly tell it to do the opposite.

I'm just pointing this out because they're not quite as 2 dimensional as you are insinuating - even if they're frequently wrong and need careful prompting for decent quality

(after the initial "you're absolutely right!" And it finished "thinking" about it)

Re: Ilya Sutskever: We're moving from the age of scaling to the age of research

#109

Post by [deleted] »

londons_explore wrote: post #76

> These models somehow just generalize dramatically worse than people. It's a very fundamental thing My guess is we'll discover that biological intelligence is 'learning' not just from your experience, but that of thousands of ancestors. There are a few weak pointers in that direction. Eg. A father who experiences a specific fear can pass that fear to grandchildren through sperm alone. [1]. I believe this is at least…

[deleted]

Re: Ilya Sutskever: We're moving from the age of scaling to the age of research

#110

Post by [deleted] »

JimmyBuckets wrote: post #106

I respect Ilya hugely as a researcher in ML and quite admire his overall humility, but I have to say I cringed quite a bit at the start of this interview when he talks about emotions, their relative complexity, and origin. Emotion is so complex, even taking all the systems in the body that it interacts with. And many mammals have very intricate socio-emotional lives - take Orcas or Elephants. There is an arrogance I…

[deleted]
Post reply on HN