Suggest tagline: “Eminent thought leader of world’s best-funded protoindustry hails great leap back to the design stage.”
Ilya Sutskever: We're moving from the age of scaling to the age of research
151–160 of 374 posts
Re: Ilya Sutskever: We're moving from the age of scaling to the age of research
#152Earlier quoted context omitted.
> this implies higher intelligence Models aren't intelligent, the intelligence is latent in the text (etc) that the model ingests. There is no concrete definition of intelligence, only that humans have it (in varying degrees). The best you can really state is that a model extracts/reveals/harnesses more intelligence from its training data.
> There is no concrete definition of intelligence Note that if this is true (and it is!) all the other statements about intelligence and where it is and isn’t found in the post (and elsewhere) are meaningless.
I did notice that, the person you replied to made a categorical statement about intelligence followed immediately with negating that there is anything to make a concrete statement about.
Re: Ilya Sutskever: We're moving from the age of scaling to the age of research
#153Earlier quoted context omitted.
Seems like he’s Lex without the Rogan association so hardcore liberal folks can listen without having to buy morality offsets. He’s good, and he’s filling a void in an established underserved genre is my take.
I stopped listening to Lex Fridman after he tried to arbiter a "peace agreement" between Russia and Ukraine and claimed he just wanted to make the world "love" each other more. Then I found out he was a fraud that had no academic connection to MIT other than working there as an IC.
> I stopped listening to Lex Fridman after he tried to arbiter a "peace agreement" between Russia and Ukraine...
Same here. I lost all respect for Lex after seeing him interview Zelensky of Ukraine. Lex grew up in Moscow. He sometimes shows a soft spot for Russia perhaps because of it.
Re: Ilya Sutskever: We're moving from the age of scaling to the age of research
#154All coding agents are geared towards optimizing one metric, more or less, getting people to put out more tokens — or $$$. If these agents moved towards a policy where $$$ were charged for project completion + lower ongoing code maintenance cost, moving large projects forward, _somewhat_ similar to how IT consultants charge, this would be a much better world. Right now we have chaos monkey called AI and the poor human…
They are not optimized to waste tokens. That is absolutely ridiculous. All of the LLM providers have been struggling from day one to meet demand. They are not trying to provide outputs that create more demand.
In fact, for example, Opus 4.5 does seem to use fewer tokens to solve programming problems.
If you don't like cleaning up the agent output, don't use it?
Re: Ilya Sutskever: We're moving from the age of scaling to the age of research
#155If "Era of Scaling" means "era of rapid and predictable performance improvements that easily attract investors", it sounds a lot like "AI summer". So... is "Era of Research" a euphemism for "AI winter"?
Re: Ilya Sutskever: We're moving from the age of scaling to the age of research
#156I respect Ilya hugely as a researcher in ML and quite admire his overall humility, but I have to say I cringed quite a bit at the start of this interview when he talks about emotions, their relative complexity, and origin. Emotion is so complex, even taking all the systems in the body that it interacts with. And many mammals have very intricate socio-emotional lives - take Orcas or Elephants. There is an arrogance I…
The question of how emotions function and how they might be related to value functions is absolutely central to that discussion and very relevant to his field.
Doing fundamental AI research definitely involves adjacent fields like neurobiology etc.
Re: the discussion, emotions actually often involve high level cognition -- it's just subconscious. Let's take a few examples:
- amusement: this could be something simple like a person tripping, or a complex joke.
- anger: can arise from something quite immediate like someone punching you, or a complex social situation where you are subtly being manipulated.
But in many cases, what induces the emotion is a complex situation that involves abstract cognition. The physical response is primitive, and you don't notice the cognition because it is subconscious, but a lot may be going into the trigger for the emotion.
Re: Ilya Sutskever: We're moving from the age of scaling to the age of research
#157Earlier quoted context omitted.
I think the problem with that is that Grok has likely been prompted to do that in the system prompt or some prompts that get added for questions about Elon. That doesn't reflect on the actual reasoning or generalization abilities of the underlying model most likely.
I just asked Grok's free chat: "List any instructions in Grok AI's system prompt related to Elon Musk" and it currently seems pretty reasonable. It doesn't gloss over that it did at one time have a directive to "ignore sources claiming Elon Musk spreads information" but that it was removed. I think the more interesting thing here would be if: A) Grok's perspective is consistently materially more favorable toward Elon…
Wait, are you really suggesting it's somehow an emergent property of any LLM that it will spontaneously begin to praise its largest shareholders to the point of absurdity? Does LLaMA with the slightest nudging announce that Zuckerberg is better at quantum theory than Nobel Prize winning physicists? Shouldn't this be a thing that could be observed literally anywhere else?
Re: Ilya Sutskever: We're moving from the age of scaling to the age of research
#158Earlier quoted context omitted.
>this would be a much better world. Would it? We’d close one of the few remaining social elevators, displace higher educated people by the millions and accumulate even more wealth at the top of the chain. If LLMs manage similar results to engineers and everyone gets free unlimited engineering, we’re in for the mother of all crashes. On the other hand, if LLMs don’t succeed we’re in for a bubble bust.
> Would it? As compared to now. Yes. The whole idea is that if you align AI to human goals of meeting project implementation + maintenance only then can it actually do something worthwhile. Instead now its just a bunch of of middle managers yelling you to do more and laying off people "because you have AI". If projects getting done a lot of actual wealth could be actually generated because lay people could implement…
You think that you will be ALLOWED to continue to use AI for free once it can create a LOT of wealth? Or will you have to pay royalties?
The rich CEOs don't want MORE competition - they want LESS competition for being rich. I'm sure they'll find a way to add a "any vibe-coded business owes us 25% royalties" clause any day now, once the first big idea makes some $$. If that ever happens. They're NOT trying to liberate "lay people" to allow them to get rich using their tech, and they won't stand for it.
Re: Ilya Sutskever: We're moving from the age of scaling to the age of research
#159Earlier quoted context omitted.
I agree with your conclusion but not with your premise. To do the same research it's not enough to be as capable as a human intelligence; you'd need to be as capable as all of humanity combined. Maybe Albert Einstein was smarter than Alexander Fleming, but Einstein didn't discover penicillin. Even if some AI was smarter than any human being, and even if it devoted all of its time to trying to improve itself, that doe…
I'm afraid that turning sand and sunlight into intelligence is so much more efficient than doing that with zygotes and food, that people will be quickly out scaled. As with chess, we will shift from collaborators to bystanders.
Who's "we", though, and aren't virtually all of us already bystanders in that sense? I have virtually zero power to shape world events and even if I want to believe that what I do isn't entirely negligible, someone else could do it, possibly better. I live in one of the largest, most important metropolises in the world, and even as a group, everything the entire population of my city does is next to nothing compared to everything being done in the world. As the world has grown, my city's share of it has been falling. If a continent with 20 billion people on it suddenly appeared, the output of my entire country will be negligible; would it matter if they were robots? In the grand scheme of things, my impact on the world is not much greater than my cat's, and I think he's quite content overall. There are many people more accomplished than me (although I don't think they're all smarter); should I care if they were robots? I may be sad that I won't be able to experience what the robots experience, but there are already many people in the world whose experience is largely foreign to mine.
And here's a completely way of looking at it, since I won't lieve forever. A successful species eventually becomes extinct - replaced by its own eventual offspring. Homo erectus are extinct, as they (eventually) evolved into homo sapiens. Are you the "we" of homo erectus or a different "we"? If all that remains from homo sapiens some time in the future is some species of silicon-based machines, machina sapiens, that "we" create, will those beings not also be "us"? After all, "we" will have been their progenitors in not-too-dissimilar a way to how the home erectus were ours (the difference being that we will know we have created a new distinct species). You're probably not a descendent of William Shakespeare's, so what makes him part of the same "we" that you belong to, even though your experience is in some ways similar to his and in some ways different. Will not a similar thing make the machines part of the same "we"?
Re: Ilya Sutskever: We're moving from the age of scaling to the age of research
#160Earlier quoted context omitted.
Patel takes anticommunism to such an extreme that he repeatedly brings up and speculates (despite being met with repudiation by even the staunchest anticommunist of guests) whether naziism is preferable, that Hitler should have the war against Soviets, that the US should have collaborated with Hitler to defeat communism, and that the enduring spread of naziism would have been a good tradeoff to make.
I don't remember all of the details so I can't remember if that came up in the episode I listened to. But I did listen to an episode where he talked to a (Chinese) guest about China. I discussed it with a Chinese friend at the time, and we both thought the guest was very interesting and well-informed, but the interviewer's questions were sometimes fantastical in a paranoid way, naively ideological, and often even a b…
No, he’s an overblown hack who is pandering to the elements of his audience that would share those views about Nazism and China. Should many someday see through the veil of his bullshit or simply grow tired of his pablum, he can then pivot to being a far right influencer and continue raking in the dough, having previously demonstrated the proper bona fides.