Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ilya Sutskever: We're moving from the age of scaling to the age of research

dwarkesh.com

Post reply on HN
Print view View original

241–250 of 374 posts

Re: Ilya Sutskever: We're moving from the age of scaling to the age of research

#241

Post by krige »

menaerus wrote: post #226
chroma205 wrote: post #214

Earlier quoted context omitted.

> I've fed thousands of dollars to Anthropic/OAI/etc for their coding models over the past year despite never having paid for dev tools before in my life. Seems commercially viable to me. For OpenAI to produce a 10% return, every iPhone user on earth needs to pay $30/month to OpenAI. That ain’t happening.

They don't sell their models to individuals only but also to companies with most likely different business and pricing models so that's an overly simplistic view of their business. YoY their spending increases, we can safely assume that one of the reasons is the growing user base. Time will probably come when we won't be allowed to consume frontier models without paying anything, as we can today, and time will come w…

>> OpenAI’s cumulative free cash flow to 2030 may be about $282bn

According to who, OpenAI? It is almost certain they flat out lie about their numbers as suggested by their 20% revenue shares with MS.

Re: Ilya Sutskever: We're moving from the age of scaling to the age of research

#242

Post by torben-friis »

itissid wrote: post #117
torben-friis wrote: post #65

Earlier quoted context omitted.

>this would be a much better world. Would it? We’d close one of the few remaining social elevators, displace higher educated people by the millions and accumulate even more wealth at the top of the chain. If LLMs manage similar results to engineers and everyone gets free unlimited engineering, we’re in for the mother of all crashes. On the other hand, if LLMs don’t succeed we’re in for a bubble bust.

> Would it? As compared to now. Yes. The whole idea is that if you align AI to human goals of meeting project implementation + maintenance only then can it actually do something worthwhile. Instead now its just a bunch of of middle managers yelling you to do more and laying off people "because you have AI". If projects getting done a lot of actual wealth could be actually generated because lay people could implement…

>If projects getting done a lot of actual wealth could be actually generated because lay people could implement things that go beyond the realm of toy projects.

Suppose LLMs create projects in the way you propose (and they don’t rug pull, which would already be rare).

Why do you think that would generate wealth for laymen? Look at music or literature, now everyone can be on Spotify or Amazon.

The result has been an absolute destruction of the wealth that reaches any author, who are buried in slop. The few that survive do so by putting 50 times more dedication into marketing than they do to the craft, any author is full time placing their content in social networks or paying to collab with artists just to be seen.

This is not an improvement for anyone. Professionals no longer make a living, laypeople have a skill that’s now useless due to offer and demand, and the sea of content favors those already positioned to create visibility - the already rich.

Re: Ilya Sutskever: We're moving from the age of scaling to the age of research

#243

Post by tim333 »

joelthelion wrote: post #227

> When do you expect that impact? I think the models seem smarter than their economic impact would imply. > Yeah. This is one of the very confusing things about the models right now. As someone who's been integrating "AI" and algorithms into people's workflows for twenty years, the answer is actually simple. It takes time to figure out how exactly to use these tools, and integrate them into existing tooling and workf…

Beyond that the smartness is very patchy. They can do math problems beyond 99% of humans but lack the common sense understanding to take over most jobs.

Re: Ilya Sutskever: We're moving from the age of scaling to the age of research

#244

Post by tim333 »

nsomaru wrote: post #238
simianparrot wrote: post #234

Earlier quoted context omitted.

Datacenter GPU's have a lifespan of 1-3 years depending on use. So yes they exist, but not for long, unless they go entirely unused. But then they also deprecate in efficiency compared to new hardware extremely fast as well, so their shelf life is severely limited either way.

Personally I am waiting for the day I can realistically buy a second hand three year old datacentre GPU so I can run Kimi K2 in my shed. Given enough time, not a pipe dream. But 10 years at least.

You'll probably be able to run Kimi K2 on the iphone 27.

Re: Ilya Sutskever: We're moving from the age of scaling to the age of research

#245

Post by ACCount37 »

tim333 wrote: post #243
joelthelion wrote: post #227

> When do you expect that impact? I think the models seem smarter than their economic impact would imply. > Yeah. This is one of the very confusing things about the models right now. As someone who's been integrating "AI" and algorithms into people's workflows for twenty years, the answer is actually simple. It takes time to figure out how exactly to use these tools, and integrate them into existing tooling and workf…

Beyond that the smartness is very patchy. They can do math problems beyond 99% of humans but lack the common sense understanding to take over most jobs.

Most jobs involve complex long term tasks - which isn't something that's natural to LLMs.

Re: Ilya Sutskever: We're moving from the age of scaling to the age of research

#246

Post by ACCount37 »

__loam wrote: post #186
orbital-decay wrote: post #132

>You could actually wonder that one possible explanation for the human sample efficiency that needs to be considered is evolution. Evolution has given us a small amount of the most useful information possible. It's definitely not small. Evolution performed a humongous amount of learning, with modern homo sapiens, an insanely complex molecular machine, as a result. We are able to learn quickly by leveraging this "pret…

Please stop comparing these things to biological systems. They have very little in common.

That's like saying that a modern calculator and a mechanical arithmometer have very little in common.

Sure, the parts are all different, and the construction isn't even remotely similar. They just happen to be doing the same thing.

Re: Ilya Sutskever: We're moving from the age of scaling to the age of research

#247

Post by menaerus »

krige wrote: post #241
menaerus wrote: post #226

Earlier quoted context omitted.

They don't sell their models to individuals only but also to companies with most likely different business and pricing models so that's an overly simplistic view of their business. YoY their spending increases, we can safely assume that one of the reasons is the growing user base. Time will probably come when we won't be allowed to consume frontier models without paying anything, as we can today, and time will come w…

>> OpenAI’s cumulative free cash flow to 2030 may be about $282bn According to who, OpenAI? It is almost certain they flat out lie about their numbers as suggested by their 20% revenue shares with MS.

A bank - HSBC. Read the article.

Re: Ilya Sutskever: We're moving from the age of scaling to the age of research

#248

Post by ACCount37 »

fzzzy wrote: post #196
tintor wrote: post #177

Earlier quoted context omitted.

So what? It knows number of letters in each token, and can sum them together.

How does it know the letters in the token? It doesn't. There's literally no mapping anywhere of the letters in a token.

There is a mapping. An internal, fully learned mapping that's derived from seeing misspellings and words spelled out letter by letter. Some models make it an explicit part of the training with subword regularization, but many don't.

It's hard to access that mapping though.

A typical LLM can semi-reliably spell common words out letter by letter - but it can't say how many of each are in a single word immediately.

But spelling the word out first and THEN counting the letters? That works just fine.

Re: Ilya Sutskever: We're moving from the age of scaling to the age of research

#249

Post by [deleted] »

aunty_helen wrote: post #51
jsheard wrote: post #4

Earlier quoted context omitted.

The translation is that SSI says that SSIs strategy is the way forward so could investors please stop giving OpenAI money and give SSI the money instead. SSI has not shown anything yet, nor does SSI intend to show anything until they have created an actual Machine God, but SSI says they can pull it off so it's all good to go ahead and wire the GDP of Norway directly to Ilya.

If we take AGI as a certainty, ie we think we can achieve AGI using silicon, then Ilya is one of the best bets you can take if you are looking to invest in this space. He has a history and he's motivated to continue working on this problem. If you think that AGI is not possible to achieve, then you probably wouldn't be giving anyone money in this space.

[deleted]

Re: Ilya Sutskever: We're moving from the age of scaling to the age of research

#250

Post by reeredfdfdf »

tim333 wrote: post #243
joelthelion wrote: post #227

> When do you expect that impact? I think the models seem smarter than their economic impact would imply. > Yeah. This is one of the very confusing things about the models right now. As someone who's been integrating "AI" and algorithms into people's workflows for twenty years, the answer is actually simple. It takes time to figure out how exactly to use these tools, and integrate them into existing tooling and workf…

Beyond that the smartness is very patchy. They can do math problems beyond 99% of humans but lack the common sense understanding to take over most jobs.

Yep, the lack of commons sense is sometimes very evident.

For instance, one of these popular generative AI services refused to remove copyright watermark from an image when asked directly. Then I told it that the image has weird text artifacts on it, and asked it to remove them. That worked perfectly.

Post reply on HN