Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ilya Sutskever: We're moving from the age of scaling to the age of research

dwarkesh.com

Post reply on HN
Print view View original

221–230 of 374 posts

Re: Ilya Sutskever: We're moving from the age of scaling to the age of research

#221

Post by wahnfrieden »

throwaway2037 wrote: post #146
camillomiller wrote: post #22

Earlier quoted context omitted.

Fridman is a morally broken grifter, who just built a persona and a brand on proven lies, claiming an association with MIT that was de facto non-existent. Not wanting to give the guy recognition is not a matter of being liberal or conservative, but just interested in truthfulness.

> claiming an association with MIT that was de facto non-existent Google search: "lex fridman and mit" Second hit: https://cces.mit.edu/team/lex-fridman/ > Lex conducts research in AI, human-robot interaction, autonomous vehicles, and machine learning at MIT.

To qualify what “conducts research” means:

> Lex does not teach any for-credit class at MIT, is not listed in the teaching faculty, and his last published research paper was published in 2018. For community outreach, Lex Fridman HAS taught classes in MIT’s IAP program, which are non-credit bearing.

> The most recent documented instance of Lex Fridman teaching an IAP class was in January 2022, when he co-instructed a series of lectures on deep learning, robotics, and AI-specialized computing hardware as part of MIT’s Independent Activities Period, scheduled from January 10 to January 14.

His profile photo btw is in front of an actual lecturer’s chalk board from a class he wasn’t involved with. The chalkboard writing is just an aesthetic. In that picture he was teaching an introductory level powerpoint about AI trends in a one-time, unpaid IAP session. That’s as authentic as it gets

Re: Ilya Sutskever: We're moving from the age of scaling to the age of research

#222

Post by NateEag »

nsoonhui wrote: post #219

One thing I’m curious about is this: Ilya Sutskever wants to build Safe Superintelligence, but he keeps his company and research very secretive. Given that building Safe Superintelligence is extraordinarily difficult — and no single person’s ideas or talents could ever be enough — how does secrecy serve that goal?

If he (or his employees) are actually exploring genuinely new, promising approaches to AGI, keeping them secret helps avoid a breakneck arms race like the one LLM vendors are currently engaged in.

Situations like that do not increase all participants' level of caution.

Re: Ilya Sutskever: We're moving from the age of scaling to the age of research

#223

Post by 4b11b4 »

nsoonhui wrote: post #219

One thing I’m curious about is this: Ilya Sutskever wants to build Safe Superintelligence, but he keeps his company and research very secretive. Given that building Safe Superintelligence is extraordinarily difficult — and no single person’s ideas or talents could ever be enough — how does secrecy serve that goal?

Doesn't sound like you listened to the interview. He addresses this and says he may make releases that would be otherwise held back because he believes it's important for developments to be seen by the public.

Re: Ilya Sutskever: We're moving from the age of scaling to the age of research

#224

Post by lovich »

zaptrem wrote: post #204
AstroBen wrote: post #191

Earlier quoted context omitted.

We still don't have a commercially viable product though?

I've fed thousands of dollars to Anthropic/OAI/etc for their coding models over the past year despite never having paid for dev tools before in my life. Seems commercially viable to me.

If you fed thousands of dollars to them, but it cost them tens of thousands of dollars in compute, it’s not commercially viable.

None of these companies have proven the unit economics on their services

Re: Ilya Sutskever: We're moving from the age of scaling to the age of research

#225

Post by photochemsyn »

Open source the training corpus.

Isn't this humanity's crown jewels? Our symbolic historical inheritance, all that those who came before us created? The net informational creation of the human species, our informational glyph, expressed as weights in a model vaster than anything yet envisionaged, a full vectorial representation of everything ever done by a historical ancestor... going right back to LUCA, the Last Universal Common Ancestor?

Really the best way to win with AI is use it to replace the overpaid executives and the parasitic shareholders and investors. Then you put all those resources into cutting edge R & D. Like Maas Biosciences. All edge. (just copy and paste into any LLM then it will be explained to you).

Re: Ilya Sutskever: We're moving from the age of scaling to the age of research

#226

Post by menaerus »

chroma205 wrote: post #214
zaptrem wrote: post #204

Earlier quoted context omitted.

I've fed thousands of dollars to Anthropic/OAI/etc for their coding models over the past year despite never having paid for dev tools before in my life. Seems commercially viable to me.

> I've fed thousands of dollars to Anthropic/OAI/etc for their coding models over the past year despite never having paid for dev tools before in my life. Seems commercially viable to me. For OpenAI to produce a 10% return, every iPhone user on earth needs to pay $30/month to OpenAI. That ain’t happening.

They don't sell their models to individuals only but also to companies with most likely different business and pricing models so that's an overly simplistic view of their business. YoY their spending increases, we can safely assume that one of the reasons is the growing user base.

Time will probably come when we won't be allowed to consume frontier models without paying anything, as we can today, and time will come when this $30 will most likely become double or triple the price.

Though the truth is that R&D around AI models, and especially their hosting (inference), is expensive and won't get any cheaper without significant algorithmic improvements. According to the history, my opinion is that we may very well be ~10 years from that moment.

EDIT: HSBC has just published some projections. From https://archive.ph/9b8Ae#selection-4079.38-4079.42

> Total consumer AI revenue will be $129bn by 2030

> Enterprise AI will be generating $386bn in annual revenue by 2030

> OpenAI’s rental costs will be a cumulative $792bn between the current year and 2030, rising to $1.4tn by 2033

> OpenAI’s cumulative free cash flow to 2030 may be about $282bn

> Squaring the first total off against the second leaves a $207bn funding hole

So, yes, expensive (mind the rental costs only) ... but forseen to be penetrating into everything imagineable.

Re: Ilya Sutskever: We're moving from the age of scaling to the age of research

#227

Post by joelthelion »

> When do you expect that impact? I think the models seem smarter than their economic impact would imply.

> Yeah. This is one of the very confusing things about the models right now.

As someone who's been integrating "AI" and algorithms into people's workflows for twenty years, the answer is actually simple. It takes time to figure out how exactly to use these tools, and integrate them into existing tooling and workflows.

Even if the models don't get any smarter, just give it a few more years and we'll see a strong impact. We're just starting to figure things out.

Re: Ilya Sutskever: We're moving from the age of scaling to the age of research

#228

Post by giardini »

energy123 wrote: post #81
Animats wrote: post #39

Earlier quoted context omitted.

It's stopped being cost-effective. Another order of magnitude of data centers? Not happening. The business question is, what if AI works about as well as it does now for the next decade or so? No worse, maybe a little better in spots. What does the industry look like? NVidia and TSMC are telling us that price/performance isn't improving through at least 2030. Hardware is not going to save us in the near term. Major i…

The ideas likely aren't vague at all given who is speaking. I'd bet they're extremely specific. Just not transparently shared with the public because it's intellectual property.

What kind of ideas would be intellectual property that was not shared? Isn't every part of LLMs, except the order of processes, publicly known ? Is there some magic algorithm previously unrevealed and held secret by a cabal of insiders?

Re: Ilya Sutskever: We're moving from the age of scaling to the age of research

#229

Post by giardini »

delichon wrote: post #138
tmp10423288442 wrote: post #82

He's talking his book. Doesn't mean he's wrong, but Dwarkesh is now big enough that you should assume every big name there is talking their book.

Here's a world class scientist here not because we had a hole in the schedule or he happened to be in town, but to discuss this subject that he thought and felt about so deeply that he had to write a book about it. That's a feature not a bug.

"Here's a world class scientist here not because we had a hole in the schedule or he happened to be in town, but to discuss this subject that he " had invested himself so fully personally and financially that, should it fail, he would be ruined.

FTFY

Re: Ilya Sutskever: We're moving from the age of scaling to the age of research

#230

Post by baq »

__loam wrote: post #186
orbital-decay wrote: post #132

>You could actually wonder that one possible explanation for the human sample efficiency that needs to be considered is evolution. Evolution has given us a small amount of the most useful information possible. It's definitely not small. Evolution performed a humongous amount of learning, with modern homo sapiens, an insanely complex molecular machine, as a result. We are able to learn quickly by leveraging this "pret…

Please stop comparing these things to biological systems. They have very little in common.

Structurally? Yes.

On the other hand, outputs of these systems are remarkably close to outputs of certain biological systems in at least some cases, so comparisons in some projections are still valid.

Post reply on HN