Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ilya Sutskever's SSI Inc raises $1B

reuters.com

Post reply on HN
Print view View original

381–390 of 803 posts

Re: Ilya Sutskever's SSI Inc raises $1B

#381

Post by lancesells »

gigatexal wrote: post #338
pembrook wrote: post #321

Earlier quoted context omitted.

While I get the cynicism (and yes, there is certainly some dumb money involved), it’s important to remember that every tech company that’s delivered 1000X returns was also seen as ridiculously overhyped/overvalued in its early days. Every. Single. One. It’s the same story with Amazon, Apple, Google, Facebook/Meta, Microsoft, etc. etc. That’s the point of venture capital; making extremely risky bets spread across a wi…

> While I get the cynicism (and yes, there is certainly some dumb money involved), it’s important to remember that every tech company that’s delivered 1000X returns was also seen as ridiculously overhyped/overvalued in its early days. Every. Single. One. It’s the same story with Amazon, Apple, Google, Facebook/Meta, Microsoft, etc. etc. Really? Selling goods online (Amazon) is not AGI. It didn’t take a huge leap to t…

I agree with what you're saying as I personally feel current AI products are almost a plugin or integration into existing software. It's a little like crypto where only a small amount of people were clamoring for it and it's a solution in search of a problem while also being a demented answer to our self-made problems like an inbox too full or the treadmill of content production.

However, I think because the money involved and all of these being forced upon us, one of these companies will get 1000x return. A perfect example is the Canva price hike from yesterday or any and every Google product from here on out. It's essentially being forced upon everyone that uses internet technology and someone is going to win while everyone else loses (consumers and small businesses).

Re: Ilya Sutskever's SSI Inc raises $1B

#382

Post by stuckkeys »

twobitshifter wrote: post #322
avocardio wrote: post #58

I don't understand how "safe" AI can raise that much money. If anything, they will have to spend double the time on red-teaming before releasing anything commercially. "Unsafe" AI seems much more profitable.

Safe super-intelligence will likely be as safe as OpenAI is open. We can’t build critical software without huge security holes and bugs (see crowdstrike) but we think we will be able to contain something smarter than us? It would only take one vulnerability.

You are not wrong. But Crowdstrike comparison is not “IT” they should have never had direct kernel access. MS set themself up for that one. SSI or whatever the hype will be in the coming future, it would be very difficult to beat. Unless of you shut down the power. It could develop guard rails instantly. So any flaw you may come up with, it would be instantly patched. Ofc this is just my take.

Re: Ilya Sutskever's SSI Inc raises $1B

#383

Post by [deleted] »

dcchambers wrote: post #336
gigatexal wrote: post #306

This being ycombinator and as such ostensibly has one or two (if not more) VCs as readers/commentators … can someone please tell me how these companies that are being invested in in the AI space are going to make returns on the money invested? What’s the business plan? (I’m not rich enough to be in these meetings) I just don’t see how the returns will happen. Open source LLMs exist and will get better. Is it just tha…

I also don't understand it. If AGI is actually reached, capital as we know it basically becomes worthless. The entire structure of the modern economy and the society surrounding it collapses overnight. I also don't think there's any way the governments of the world let real AGI stay in the hands of private industry. If it happens, governments around the world will go to war to gain control of it. SSI would be nationa…

[deleted]

Re: Ilya Sutskever's SSI Inc raises $1B

#384

Post by hn_throwaway_99 »

famouswaffles wrote: post #361
hn_throwaway_99 wrote: post #267

Earlier quoted context omitted.

Thank you very much for posting! This is exactly what I was looking for. On one hand, I understand what he's saying, and that's why I have been frustrated in the past when I've heard people say "it's just fancy autocomplete" without emphasizing the awesome capabilities that can give you. While I haven't seen this video by Sutskever before, I have seen a very similar argument by Hinton: in order to get really good at…

>All that said, I find his argument wholly unconvincing (and again, I may be waaaaay stupider than Sutskever, but there are other people much smarter than I who agree). And the reason for this is because every now and then I'll see a particular type of hallucination where it's pretty obvious that the LLM is confusing similar token strings even when their underlying meaning is very different. That is, the underlying "…

You are completely mischaracterizing my comment.

> Humans have a long list of cognitive shortcomings. We find them interesting and give them all sorts of names like cognitive dissonance or optical illusions. But we don't currently make silly conclusions like humans don't reason.

Exactly! In fact, things like illusions are actually excellent windows into how the mind really works. Most visual illusions are a fundamental artifact of how the brain needs to turn a 2D image into a 3D, real-world model, and illusions give clues into how it does that, and how the contours of the natural world guided the evolution of the visual system (I think Steven Pinker's "How the Mind Works" gives excellent examples of this).

So I am not at all saying that what LLMs do isn't extremely interesting, or useful. What I am saying is that the types of errors you get give a window into how an LLM works, and these hint at some fundamental limitations at what an LLM is capable of, particularly around novel discovery and development of new ideas and theories that aren't just "rearrangements" of existing ideas.

Re: Ilya Sutskever's SSI Inc raises $1B

#385

Post by HarHarVeryFunny »

famouswaffles wrote: post #376
HarHarVeryFunny wrote: post #367

Earlier quoted context omitted.

> A transformer can perform any computation it likes in a forward pass No it can't. A transformer has a fixed number of layers - call it N. It performs N sequential steps of computation to derive it's output. If a computation requires > N steps, then a transformer most certainly can not perform it in a forward pass. FYI, "attention is all you need" has the implicit context of "if all you want to build is a language m…

Transformer produce the next token by manipulating K hidden vectors per layer, one vector per preceding token. So yes you can increase compute length arbitrarily by increasing tokens. Those tokens don't have to carry any information to work. https://arxiv.org/abs/2310.02226 And again, human brains are clearly limited in the number of steps it can compute without writing something down. Limited =/ Trivial >FYI, "atten…

You are confusing number of sequential steps with total amount of compute spent.

The input sequence is processed in parallel, regardless of length, so number of tokens has no impact on number of sequential compute steps which is always N=layers.

> Do you know what a "language model" is capable of in the limit ?

Well, yeah, if the language model is an N-layer transformer ...

Re: Ilya Sutskever's SSI Inc raises $1B

#386

Post by davedx »

gigatexal wrote: post #338
pembrook wrote: post #321

Earlier quoted context omitted.

While I get the cynicism (and yes, there is certainly some dumb money involved), it’s important to remember that every tech company that’s delivered 1000X returns was also seen as ridiculously overhyped/overvalued in its early days. Every. Single. One. It’s the same story with Amazon, Apple, Google, Facebook/Meta, Microsoft, etc. etc. That’s the point of venture capital; making extremely risky bets spread across a wi…

> While I get the cynicism (and yes, there is certainly some dumb money involved), it’s important to remember that every tech company that’s delivered 1000X returns was also seen as ridiculously overhyped/overvalued in its early days. Every. Single. One. It’s the same story with Amazon, Apple, Google, Facebook/Meta, Microsoft, etc. etc. Really? Selling goods online (Amazon) is not AGI. It didn’t take a huge leap to t…

Imagine empowering accountants and all other knowledge workers, on steroids, drastically simplifying all their day to day tasks and reducing them to purely executive functions.

Imagine organizing the world's data and knowledge, and integrating it seamlessly into every possible workflow.

Now you're getting close.

But also remember, this company is not trying to produce AGI (intelligence comparable to the flexibility of human cognition), it's trying to produce super intelligence (intelligence beyond human cognition). Imagine what that could do for your job, career, dreams, aspirations, moon shots.

Re: Ilya Sutskever's SSI Inc raises $1B

#387

Post by qingcharles »

tim333 wrote: post #356
jedberg wrote: post #339

Earlier quoted context omitted.

It's more game theory. Regardless of the chances of AGI, if you're not invested in it, you will lose everything if it happens. It's more like a hedge on a highly unlikely event. Like insurance. And we already seeing a ton of value in LLMs. There are lots of companies that are making great use of LLMs and providing a ton of value. One just launched today in fact: https://www.paradigmai.com/ (I'm an investor in that).…

If you want safe investment you could always buy land. AGI won't be able to make more of that.

If ASI arrives we'll need a fraction of the land we use already. We'll all disappear into VR pods hooked to a singularity metaverse and the only sustenance we'll need is some Soylent Green style sludge that the ASI will make us believe tastes like McRib(tm).

Re: Ilya Sutskever's SSI Inc raises $1B

#388

Post by HarHarVeryFunny »

famouswaffles wrote: post #376
HarHarVeryFunny wrote: post #367

Earlier quoted context omitted.

> A transformer can perform any computation it likes in a forward pass No it can't. A transformer has a fixed number of layers - call it N. It performs N sequential steps of computation to derive it's output. If a computation requires > N steps, then a transformer most certainly can not perform it in a forward pass. FYI, "attention is all you need" has the implicit context of "if all you want to build is a language m…

Transformer produce the next token by manipulating K hidden vectors per layer, one vector per preceding token. So yes you can increase compute length arbitrarily by increasing tokens. Those tokens don't have to carry any information to work. https://arxiv.org/abs/2310.02226 And again, human brains are clearly limited in the number of steps it can compute without writing something down. Limited =/ Trivial >FYI, "atten…

> And again, human brains are clearly limited in the number of steps it can compute without writing something down

No - there is a loop between the cortex and thalamus, feeding the outputs of the cortex back in as inputs. Our brain can iterate for as long as it likes before initiating any motor output, if any, such as writing something down.

Re: Ilya Sutskever's SSI Inc raises $1B

#389

Post by davedx »

aithrowaway1987 wrote: post #152
davedx wrote: post #70

Earlier quoted context omitted.

> There is significant possibility that true AI (what Ilia calls superintelligence) is impossible to build using neural networks What evidence can you provide to back up the statement of this "significant possibility"? Human brains use neural networks...

There was a very good paper in Nature showing this definitively: https://news.ycombinator.com/item?id=41437933 Modern ANN architectures are not actually capable of long-term learning in the same way animals are, even stodgy old dogs that don't learn new tricks. ANNs are not a plausible model for the brain, even if they emulate certain parts of the brain (the cerebellum, but not the cortex) I will add that transformer…

> Modern ANN architectures are not actually capable of long-term learning

What do you think training (and fine-tuning) does?

Re: Ilya Sutskever's SSI Inc raises $1B

#390

Post by davedx »

waveBidder wrote: post #93
davedx wrote: post #70

Earlier quoted context omitted.

> There is significant possibility that true AI (what Ilia calls superintelligence) is impossible to build using neural networks What evidence can you provide to back up the statement of this "significant possibility"? Human brains use neural networks...

no, there's really no comparing barely nonlinear algrebra that makes up transformers and the tangled mess that is human neurons. the name is an artifact and a useful bit of salesmanship.

Sure, it's a model. But don't we think neural networks and human brains are primarily about their connectedness and feedback mechanisms though?

(I did AI and Psychology at degree level, I understand there are definitely also big differences too, like hormones and biological neurones being very async)

Post reply on HN