Ilya Sutskever's SSI Inc raises $1B • Hacker News Board

Re: Ilya Sutskever's SSI Inc raises $1B

#271

Post by ldjkfkdsjnv » Wed, Sep 04, 2024, 5:14 PM UTC

hn_throwaway_99 wrote: post #250

nilkn wrote: post #231

Earlier quoted context omitted.

The argument about AGI from LLMs is not based on the current state of LLMs, but on the rate of progress over the last 5+ years or so. It wasn't very long ago that almost nobody outside of a few niche circles seriously thought LLMs could do what they do right now. That said, my personal hypothesis is that AGI will emerge from video generation models rather than text generation models. A model that takes an arbitrary r…

> The argument about AGI from LLMs is not based on the current state of LLMs, but on the rate of progress over the last 5+ years or so. And what I'm saying is that I find that argument to be incredibly weak. I've seen it time and time again, and honestly at this point just feels like a "humans should be a hundred feet tall based on on their rate of change in their early years" argument. While I've also been amazed at…

10 years of progress is a flash in the pan of human progress. The first deep learning models that worked appeared in 2012. That was like yesterday. You are completely underestimating the rate of change we are witnessing. Compute scaling is not at all similar to biological scaling.

Re: Ilya Sutskever's SSI Inc raises $1B

#272

Post by bschmidt1 » Wed, Sep 04, 2024, 5:15 PM UTC

whimsicalism wrote: post #128

sim7c00 wrote: post #104

Earlier quoted context omitted.

well since it's no longer ok to just suck up anyone's data and train your AI, it will be a new challenge for them to avoid that pitfall. I can imagine it will take some time...

what laws have actually changed that make it no longer okay? we all know that openai did it

There are class actions now like https://www.nytimes.com/2024/06/13/business/clearview-ai-fac...

Nobody even knew what OpenAI was up to when they were gathering training data - they got away with a lot. Now there is precedent and people are paying more attention. Data that was previously free/open now has a clause that it can't be used for AI training. OpenAI didn't have to deal with any of that.

Also OpenAI used cheap labor in Africa to tag training data which was also controversial. If someone did it now it would they'd be the ones to pay. OpenAI can always say "we stopped" like Nike said with sweat shops.

A lot has changed.

Re: Ilya Sutskever's SSI Inc raises $1B

#273

Post by vasco » Wed, Sep 04, 2024, 5:16 PM UTC

zombiwoof wrote: post #263

mitthrowaway2 wrote: post #243

Earlier quoted context omitted.

I'd put decent odds on a $1B research project developing time travel if time travel were an ability that every human child was innately born with. It's never easy to recreate what biology has done, but nature providing an "existence proof" goes a long way towards removing doubt about it being fundamentally possible.

Nature didn’t build intelligence with non biological activity. And we won’t either

"Biological activity" is just computation with different energy requirements. If science rules the universe we're complex automata, and biologic machines or non-biological machines are just different combinations of atoms that are computing around.

Re: Ilya Sutskever's SSI Inc raises $1B

#274

Post by otabdeveloper4 » Wed, Sep 04, 2024, 5:18 PM UTC

davedx wrote: post #70

Yizahi wrote: post #47

Earlier quoted context omitted.

There is significant possibility that true AI (what Ilia calls superintelligence) is impossible to build using neural networks. So it is closer to some tokenbro project than to nuclear research. Or he will simply shift goalposts, and call some LLM superintelligent.

> There is significant possibility that true AI (what Ilia calls superintelligence) is impossible to build using neural networks What evidence can you provide to back up the statement of this "significant possibility"? Human brains use neural networks...

> Human brains use neural networks...

They don't, actually.

Re: Ilya Sutskever's SSI Inc raises $1B

#275

Post by nilkn » Wed, Sep 04, 2024, 5:19 PM UTC

ldjkfkdsjnv wrote: post #270

nilkn wrote: post #231

Earlier quoted context omitted.

The argument about AGI from LLMs is not based on the current state of LLMs, but on the rate of progress over the last 5+ years or so. It wasn't very long ago that almost nobody outside of a few niche circles seriously thought LLMs could do what they do right now. That said, my personal hypothesis is that AGI will emerge from video generation models rather than text generation models. A model that takes an arbitrary r…

If its true that predicting the next word can be turned into predict the next pixel. And that you could run a zillion hours of video feed into that, I agree. It seems that the basic algorithm is there. Video is much less information dense than text, but if the scale of compute can reach the 10s of billions of dollars, or more, you have to expect that AGI is achievable. I think we will see it in our lifetimes. Its pro…

I feel like that's already been demonstrated with the first-generation video generation models we're seeing. Early research already shows video generation models can become world simulators. There frankly just isn't enough compute yet to train models large enough to do this for all general phenomena and then make it available to general users. It's also unclear if we have enough training data.

Video is not necessarily less information dense than text, because when considered in its entirety it contains text and language generation as special cases. Video generation includes predicting continuations of complex verbal human conversations as well as continuations of videos of text exchanges, someone flipping through notes or a book, someone taking a university exam through their perspective, etc.

Re: Ilya Sutskever's SSI Inc raises $1B

#276

Post by nilkn » Wed, Sep 04, 2024, 5:20 PM UTC

hn_throwaway_99 wrote: post #250

nilkn wrote: post #231

Earlier quoted context omitted.

The argument about AGI from LLMs is not based on the current state of LLMs, but on the rate of progress over the last 5+ years or so. It wasn't very long ago that almost nobody outside of a few niche circles seriously thought LLMs could do what they do right now. That said, my personal hypothesis is that AGI will emerge from video generation models rather than text generation models. A model that takes an arbitrary r…

> The argument about AGI from LLMs is not based on the current state of LLMs, but on the rate of progress over the last 5+ years or so. And what I'm saying is that I find that argument to be incredibly weak. I've seen it time and time again, and honestly at this point just feels like a "humans should be a hundred feet tall based on on their rate of change in their early years" argument. While I've also been amazed at…

Expecting the rate of progress to drop off so abruptly after realistically just a few years of serious work on the problem seems like the more unreasonable and grander prediction to me than expecting it to continue at its current pace for even just 5 more years.

Re: Ilya Sutskever's SSI Inc raises $1B

#277

Post by JamesSwift » Wed, Sep 04, 2024, 5:21 PM UTC

hn_throwaway_99 wrote: post #267

petulla wrote: post #213

Earlier quoted context omitted.

Ilya has discussed this question: https://www.youtube.com/watch?v=YEUclZdj_Sc

Thank you very much for posting! This is exactly what I was looking for. On one hand, I understand what he's saying, and that's why I have been frustrated in the past when I've heard people say "it's just fancy autocomplete" without emphasizing the awesome capabilities that can give you. While I haven't seen this video by Sutskever before, I have seen a very similar argument by Hinton: in order to get really good at…

Watching that video actually makes me completely unconvinced that SSI will succeed if they are hinging it on LLM...

He puts a lot of emphasis on the fact that 'to generate the next token you must understand how', when thats precisely the parlor trick that is making people lose their minds (myself included) with how effective current LLMs are. The fact that it can simulate some low-fidelity reality with _no higher-level understanding of the world_, using purely linguistic/statistical analysis, is mind-blowing. To say "all you have to do is then extrapolate" is the ultimate "draw the rest of the owl" argument.

Re: Ilya Sutskever's SSI Inc raises $1B

#278

Post by scarmig » Wed, Sep 04, 2024, 5:22 PM UTC

aithrowaway1987 wrote: post #226

whimsicalism wrote: post #177

Earlier quoted context omitted.

this paper is far from “showing this definitively” even if we bought this negative result as somehow “proving impossibility”, i’m not convinced plasticity is necessary for intelligence huge respect for richard sutton though

Isn't "plasticity is not necessary for intelligence" just defining intelligence downwards? It seems like you want to restrict "intelligence" to static knowledge and (apparent) short-term cleverness, but being able to make long-term observation and judgements about a changing world is a necessary component of intelligence in vertebrates. Why exclude that from consideration? More specifically: it is highly implausible…

Anterograde amnesia is a significant disruption of plasticity, and yet people who have it are still intelligent.

(That said, I agree plasticity is key to the most powerful systems. A human race with anterograde amnesia would have long ago gone extinct.)

Re: Ilya Sutskever's SSI Inc raises $1B

#279

Post by twobitshifter » Wed, Sep 04, 2024, 5:22 PM UTC

Yizahi wrote: post #47

xianshou wrote: post #19

Same funding as OpenAI when they started, but SSI explicitly declared their intention not to release a single product until superintelligence is reached. Closest thing we have to a Manhattan Project in the modern era?

There is significant possibility that true AI (what Ilia calls superintelligence) is impossible to build using neural networks. So it is closer to some tokenbro project than to nuclear research. Or he will simply shift goalposts, and call some LLM superintelligent.

Maybe closer to energy positive fusion?

Re: Ilya Sutskever's SSI Inc raises $1B

#280

Post by paxys » Wed, Sep 04, 2024, 5:24 PM UTC

Quinner wrote: post #80

paxys wrote: post #26

Earlier quoted context omitted.

> Closest thing we have to a Manhattan Project in the modern era? Minus the urgency, scientific process, well-defined goals, target dates, public ownership, accountability...

If public ownership means we give one guy a button to end the world, I'm not sure how's that's a meaningful difference.

The fact that the world hasn't ended and no nuke has been launched since the 1940s shows that the system is working. Give the button to a random billionaire and half of us will be dead by next week to improve profit margins.