Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ilya Sutskever's SSI Inc raises $1B

reuters.com

Post reply on HN
Print view View original

361–370 of 803 posts

Re: Ilya Sutskever's SSI Inc raises $1B

#361

Post by famouswaffles »

hn_throwaway_99 wrote: post #267
petulla wrote: post #213

Earlier quoted context omitted.

Ilya has discussed this question: https://www.youtube.com/watch?v=YEUclZdj_Sc

Thank you very much for posting! This is exactly what I was looking for. On one hand, I understand what he's saying, and that's why I have been frustrated in the past when I've heard people say "it's just fancy autocomplete" without emphasizing the awesome capabilities that can give you. While I haven't seen this video by Sutskever before, I have seen a very similar argument by Hinton: in order to get really good at…

>All that said, I find his argument wholly unconvincing (and again, I may be waaaaay stupider than Sutskever, but there are other people much smarter than I who agree). And the reason for this is because every now and then I'll see a particular type of hallucination where it's pretty obvious that the LLM is confusing similar token strings even when their underlying meaning is very different. That is, the underlying "pattern matching" of LLMs becomes apparent in these situations.

So? One of the most frustrating parts of these discussions is that for some bizzare reason, a lot of people have a standard of reasoning (for machines) that only exists in fiction or their own imaginations.

Humans have a long list of cognitive shortcomings. We find them interesting and give them all sorts of names like cognitive dissonance or optical illusions. But we don't currently make silly conclusions like humans don't reason.

The general reasoning engine that makes neither mistake nor contradiction or confusion in output or process does not exist in real life whether you believe Humans are the only intelligent species on the planet or are gracious enough to extend the capability to some of our animal friends.

So the LLM confuses tokens every now and then. So what ?

Re: Ilya Sutskever's SSI Inc raises $1B

#362

Post by whiplash451 »

Ilya is basically building the Tandem Computers of AI.

Before Tandem, computers used to fail regularly. Tandem changed that forever (with a massive reward for their investors).

Similarly, LLMs are known to fail regularly. Until someone figures out a way for them not to hallucinate anymore. Which is exactly what Ilya is after.

Re: Ilya Sutskever's SSI Inc raises $1B

#363

Post by tim333 »

gigatexal wrote: post #334
jedberg wrote: post #330

Earlier quoted context omitted.

The TMV (Total Market Value) of solving AGI is infinity. And furthermore, if AGI is solved, the TMV of pretty much everything else drops to zero. The play here is to basically invest in all possible players who might reach AGI, because if one of them does, you just hit the infinite money hack. And maybe with SSI you've saved the world too.

So then the investment thesis hinges on what the investor thinks AGI’s chances are. 1/100 1/1M 1/1T? What if it never pans out is there infrastructure or other ancillary tech that society could benefit from? For example all the science behind the LHC, or bigger and better telescopes: we might never find the theory of everything but the tech that goes into space travel, the science of storing and processing all that d…

AGI is likely but whether Ilya Sutskever will get there first or get the value is questionable. I kind of hope things will end up open source with no one really owning it.

Re: Ilya Sutskever's SSI Inc raises $1B

#364

Post by Vecr »

ryan93 wrote: post #360
Vecr wrote: post #95

Earlier quoted context omitted.

There are algorithms that should work, they're just galactic[0] or are otherwise expected to use far too much space and time to be practical. [0]: https://en.wikipedia.org/wiki/Galactic_algorithm

That wiki article has nothing to do with AI. The whole AI space attracts BS talk

What do you think AI is? On that one page there's simulated annealing with a logarithmic cooling schedule, Hutter search, and Solomonoff induction, all very much applicable to AI. If you want a fully complete galactic algorithm for AI, look up AIXItl.

Edit: actually I'm not sure if AIXItl is technically galactic or just terribly inefficient, but there's been trouble making it faster and more compact.

Re: Ilya Sutskever's SSI Inc raises $1B

#365

Post by digging »

subsubzero wrote: post #307
digging wrote: post #145

Earlier quoted context omitted.

Interesting attributes to mention... The urgency was faked and less true of the Manhattan Project than it is of AGI safety. There was no nuclear weapons race; once it became clear that Germany had no chance of building atomic bombs, several scientists left the MP in protest, saying it was unnecessary and dangerous. However, the race to develop AGI is very real, and we also have no way of knowing how close anyone is t…

I agree and also disagree. > There was no nuclear weapons race; once it became clear that Germany had no chance of building atomic bombs, several scientists left the MP in protest You are forgetting Japan in WWII and given casualty numbers from island hopping it was going to be a absolutely huge casualty count with US troops, probably something on the order of Englands losses during WW1. Which for them sent them on a…

Did you stop reading my comment there? I debunked this already.

Re: Ilya Sutskever's SSI Inc raises $1B

#366

Post by tim333 »

hn_throwaway_99 wrote: post #200

Lots of comments either defending this ("it's taking a chance on being the first to build AGI with a proven team") or saying "it's a crazy valuation for a 3 month old startup". But both of these "sides" feel like they miss the mark to me. On one hand, I think it's great that investors are willing to throw big chunks of money at hard (or at least expensive) problems. I'm pretty sure all the investors putting money in…

>"We’ve identified a new mountain to climb that’s a bit different from what I was working on previously. We’re not trying to go down the same path faster. If you do something different, then it becomes possible for you to do something special."

Doesn't really imply let's just do more LLMs.

Re: Ilya Sutskever's SSI Inc raises $1B

#367

Post by HarHarVeryFunny »

famouswaffles wrote: post #319
HarHarVeryFunny wrote: post #288

Earlier quoted context omitted.

No amount of training would cause a fly brain to be able to do what an octopus or bird brain can, or to model their behavioral generating process. No amount of training will cause a transformer to magically sprout feedback paths or internal memory, or an ability to alter it's own weights, etc. Architecture matters. The best you can hope for an LLM is that training will converge on the best LLM generating process it c…

>No amount of training would cause a fly brain to be able to do what an octopus or bird brain can, or to model their behavioral generating process. Go back a few evolutionary steps and sure you can. Most ANN architectures basically have relatively little to no biases baked in and the Transformer might be the most blank slate we've built yet. >No amount of training will cause a transformer to magically sprout feedback…

> A transformer can perform any computation it likes in a forward pass

No it can't.

A transformer has a fixed number of layers - call it N. It performs N sequential steps of computation to derive it's output.

If a computation requires > N steps, then a transformer most certainly can not perform it in a forward pass.

FYI, "attention is all you need" has the implicit context of "if all you want to build is a language model". Attention is not all you need if what you actually want to build is a cognitive architecture.

Re: Ilya Sutskever's SSI Inc raises $1B

#368

Post by barrell »

shwaj wrote: post #292
barrell wrote: post #136

Earlier quoted context omitted.

I agree with your stance - that being said there aren’t two options, one being identical or radically different. It’s not even a gradient between two choices, because there are several dimensions involved and nobody even knows what Superintelligence is anyways. If you wanted to reduce it down, I would say there are two possibilities: 1. Our understanding of Neurel Nets is currently sufficient to recreate intelligence…

There are layers of abstraction on top of “the math”. The back propagation math for a transformer is no different than for a multi-layer perception, yet a transformer is vastly more capable than a MLP. More to the point, it took a series of non-trivial steps to arrive at the transformer architecture. In other words, understanding the lowest-level math is no guarantee that you understand the whole thing, otherwise the…

I don’t disagree that it’s non-trivial, but we’re comparing this to conciousness, intelligence, even life. Personally I think it’s apples and an orange grove, but I guess we’ll get our answer eventually. Pretty sure we’re on the path to take transformers to their limit, wherever that may be

Re: Ilya Sutskever's SSI Inc raises $1B

#369

Post by lupire »

reducesuffering wrote: post #345
Yizahi wrote: post #47

Earlier quoted context omitted.

There is significant possibility that true AI (what Ilia calls superintelligence) is impossible to build using neural networks. So it is closer to some tokenbro project than to nuclear research. Or he will simply shift goalposts, and call some LLM superintelligent.

The only goalposts shifting are the ones who think completely blowing past the Turing Test, unlocking recursive exponential code generation, and a computer passing all the college standard tests (our way of determining human intelligence to go Harvard/MIT) better than 99% of humans, isn't a very big deal.

Funny how a human can learn to do those things with approximately $1B less effort.

Re: Ilya Sutskever's SSI Inc raises $1B

#370

Post by lupire »

signatoremo wrote: post #320
torginus wrote: post #103

Earlier quoted context omitted.

A non-cynical take is that Ilya wanted to do research without the pressure of having to release a marketable product and figuring out how to monetize their technology, which is why he left OpenAI. A very cynical take is that this is an extreme version of 'we plan to spend all money on growth and figure out monetization later' model that many social media companies with a burn rate of billions of $$, but no business m…

He was on the record that their first product will be a safe superintelligence and it won’t do anything else until then, which sounds like they won’t have paid customers until they can figure out how to build a superintelligent model. That’s certainly a lofty goal and a very long term play.

OpenAI was "on the record" with a lot of obsolete claims too. Money changes people.
Post reply on HN