Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ilya Sutskever's SSI Inc raises $1B

reuters.com

Post reply on HN
Print view View original

351–360 of 803 posts

Re: Ilya Sutskever's SSI Inc raises $1B

#351

Post by mgfist »

EGreg wrote: post #327
mgfist wrote: post #234

Earlier quoted context omitted.

I never understood this line of reasoning, because it presumes that everyone should have access to the same opportunities. It's clearly silly once you throw a few counter examples: should a Private in the military be able to skip the ranks and be promoted straight to General? Should a new grad software dev be able to be promoted to lead engineer without getting any experience? Clearly there are reasons why opportunit…

Presumably the private and the general are in the SAME organization and yes, the avenues for advancement are available equally to all, it’s based on merit and the rules are clear. The analogy would be if the private could become a major overnight because they knew a guy.

Yes but a private cannot become a general without decades of experience.

What we see with ilya is not dissimilar. I don't see why it's bad that people are more hesitant to give a talented 18 year old $1B than the guy who's been at the forefront of AI innovation.

Re: Ilya Sutskever's SSI Inc raises $1B

#352

Post by machiaweliczny »

dontlikeyoueith wrote: post #215
aithrowaway1987 wrote: post #115

Earlier quoted context omitted.

In 2022 Ilya Sutskever claimed there wasn't a distinction: > It may look—on the surface—that we are just learning statistical correlations in text. But it turns out that to ‘just learn’ the statistical correlations in text, to compress them really well, what the neural network learns is some representation of the process that produced the text. This text is actually a projection of the world. ( https://www.youtube.co…

> But it turns out that to ‘just learn’ the statistical correlations in text, to compress them really well, what the neural network learns is some representation of the process that produced the text This is pretty sloppy thinking. The neural network learns some representation of a process that COULD HAVE produced the text. (this isn't some bold assertion, it's just the literal definition of a statistical model). The…

YeS, exactly. The trick is to have enough tough data so you find optimal one. I think as we will scale models back to smaller sizes we will discover viable/correct representations

Re: Ilya Sutskever's SSI Inc raises $1B

#353

Post by tim333 »

dcchambers wrote: post #336
gigatexal wrote: post #306

This being ycombinator and as such ostensibly has one or two (if not more) VCs as readers/commentators … can someone please tell me how these companies that are being invested in in the AI space are going to make returns on the money invested? What’s the business plan? (I’m not rich enough to be in these meetings) I just don’t see how the returns will happen. Open source LLMs exist and will get better. Is it just tha…

I also don't understand it. If AGI is actually reached, capital as we know it basically becomes worthless. The entire structure of the modern economy and the society surrounding it collapses overnight. I also don't think there's any way the governments of the world let real AGI stay in the hands of private industry. If it happens, governments around the world will go to war to gain control of it. SSI would be nationa…

I think it would be much less dramatic than that if you mean human level abilities by AGI. Initially you might be able to replace the odd human by a robot equivalent probably costing more to begin with. To scale to replace everyone levels would take years and life would probably go on as normal for quite a while. Down the line assuming lots of ASI robots, if you wanted them to farm or build you a house say you'd still need land, materials, compute and energy which will not be unlimited.

Re: Ilya Sutskever's SSI Inc raises $1B

#354

Post by machiaweliczny »

HarHarVeryFunny wrote: post #288
famouswaffles wrote: post #251

Earlier quoted context omitted.

It's not sloppy. It just doesn't matter in the limit of training. 1. An Octopus and a Raven have wildly different brains. Both are intelligent. So just the idea that there is some "one true system" that the NN must discover or converge on is suspect. Even basic arithmetic has numerous methods. 2. In the limit of training on a diverse dataset (ie as val loss continues to go down), it will converge on the process (what…

No amount of training would cause a fly brain to be able to do what an octopus or bird brain can, or to model their behavioral generating process. No amount of training will cause a transformer to magically sprout feedback paths or internal memory, or an ability to alter it's own weights, etc. Architecture matters. The best you can hope for an LLM is that training will converge on the best LLM generating process it c…

How about spiders intelligence? They don’t even have brain

Re: Ilya Sutskever's SSI Inc raises $1B

#355

Post by svnt »

hn_throwaway_99 wrote: post #331
nilkn wrote: post #276

Earlier quoted context omitted.

Expecting the rate of progress to drop off so abruptly after realistically just a few years of serious work on the problem seems like the more unreasonable and grander prediction to me than expecting it to continue at its current pace for even just 5 more years.

The problem is that the rate of progress over the past 5/10/15 years has not been linear at all, and it's been pretty easy to point out specific inflection points that have allowed that progress to occur. I.e. the real breakthrough that allowed such rapid progress was transformers in 2017. Since that time, the vast majority of the progress has simply been to throw more data at the problem, and to make the models bigg…

Transformers in 2017 as the basis, but then the quantization-emergence link as a grad student project using spare time on ridiculously large A100 clusters in 2021/2022 is what finally brought about this present moment.

I feel it is fair to say that neither of these were natural extrapolations from prior successful models directly. There is no indication we are anywhere near another nonlinearity, if we even knew how to look for that.

Blind faith in extrapolation is a finance regime, not an engineering regime. Engineers encounter nonlinearities regularly. Financiers are used to compound interest.

Re: Ilya Sutskever's SSI Inc raises $1B

#356

Post by tim333 »

jedberg wrote: post #339
gigatexal wrote: post #334

Earlier quoted context omitted.

So then the investment thesis hinges on what the investor thinks AGI’s chances are. 1/100 1/1M 1/1T? What if it never pans out is there infrastructure or other ancillary tech that society could benefit from? For example all the science behind the LHC, or bigger and better telescopes: we might never find the theory of everything but the tech that goes into space travel, the science of storing and processing all that d…

It's more game theory. Regardless of the chances of AGI, if you're not invested in it, you will lose everything if it happens. It's more like a hedge on a highly unlikely event. Like insurance. And we already seeing a ton of value in LLMs. There are lots of companies that are making great use of LLMs and providing a ton of value. One just launched today in fact: https://www.paradigmai.com/ (I'm an investor in that).…

If you want safe investment you could always buy land. AGI won't be able to make more of that.

Re: Ilya Sutskever's SSI Inc raises $1B

#357

Post by whiplash451 »

gigatexal wrote: post #306

This being ycombinator and as such ostensibly has one or two (if not more) VCs as readers/commentators … can someone please tell me how these companies that are being invested in in the AI space are going to make returns on the money invested? What’s the business plan? (I’m not rich enough to be in these meetings) I just don’t see how the returns will happen. Open source LLMs exist and will get better. Is it just tha…

The company that builds the best LLM will reap dozens or hundreds of billions in reward. It’s that simple.

It has nothing to do with AGI and everything to do with being the first-party provider for Microsoft and the like.

Re: Ilya Sutskever's SSI Inc raises $1B

#358

Post by jedberg »

tim333 wrote: post #356
jedberg wrote: post #339

Earlier quoted context omitted.

It's more game theory. Regardless of the chances of AGI, if you're not invested in it, you will lose everything if it happens. It's more like a hedge on a highly unlikely event. Like insurance. And we already seeing a ton of value in LLMs. There are lots of companies that are making great use of LLMs and providing a ton of value. One just launched today in fact: https://www.paradigmai.com/ (I'm an investor in that).…

If you want safe investment you could always buy land. AGI won't be able to make more of that.

We can already make more land. See Dubai for example. And with AGI, I suspect we could rapidly get to space travel to other planets or more efficient use of our current land.

In fact I would say that one of the things that goes to values near zero would be land if AGI exists.

Re: Ilya Sutskever's SSI Inc raises $1B

#359

Post by Satam »

petulla wrote: post #213
hn_throwaway_99 wrote: post #200

Lots of comments either defending this ("it's taking a chance on being the first to build AGI with a proven team") or saying "it's a crazy valuation for a 3 month old startup". But both of these "sides" feel like they miss the mark to me. On one hand, I think it's great that investors are willing to throw big chunks of money at hard (or at least expensive) problems. I'm pretty sure all the investors putting money in…

Ilya has discussed this question: https://www.youtube.com/watch?v=YEUclZdj_Sc

To clarify this, I think it's reasonable that token prediction as a training objective could lead to AGI given the underlying model has the correct architecture. The question really is if the underlying architecture is good enough to capitalize on the training objective so as to result in superhuman intelligence.

For example, you'll have little luck achieving AGI with decision trees no matter what's their training objective.

Re: Ilya Sutskever's SSI Inc raises $1B

#360

Post by ryan93 »

Vecr wrote: post #95
Yizahi wrote: post #72

Earlier quoted context omitted.

Theoretical foundation was slowly built over decades before it started though. And correct me if I'm wrong, but calculations that it was feasible were present before the start too. They had to calculate how to do it, what will be the processes, how to construct it and so on, but theoretically scientists knew that this amount of material can start such process. On the other hand not only there is no clear path to AI t…

There are algorithms that should work, they're just galactic[0] or are otherwise expected to use far too much space and time to be practical. [0]: https://en.wikipedia.org/wiki/Galactic_algorithm

That wiki article has nothing to do with AI. The whole AI space attracts BS talk
Post reply on HN