Ilya Sutskever's SSI Inc raises $1B • Hacker News Board

Re: Ilya Sutskever's SSI Inc raises $1B

#371

Post by throwawayk7h » Wed, Sep 04, 2024, 7:43 PM UTC

gigatexal wrote: post #316

throwawayk7h wrote: post #313

Earlier quoted context omitted.

If Ilya is sincere in his belief about safe superintelligence being within reach in a decade or so, and the investors sincerely believe this as well, then the business plan is presumably to deploy the superintelligence in every field imaginable. "SSI" in pharmaceuticals alone would be worth the investment. It could cure every disease humanity has ever known, which should give it at least a $2 trillion valuation. I'm…

I’m dubious about super intelligence. Maybe I’ve seen one too many sci-fi dystopian films but I guess yes, iif it can be done and be safe sure it’d be worth trillions.

I am dubious that it can realistically be done safely. However, we shouldn't let sci-fi films with questionable interpretations of time travel cloud our judgment, even if they are classics that we adore.

Re: Ilya Sutskever's SSI Inc raises $1B

#372

Post by obmelvin » Wed, Sep 04, 2024, 7:44 PM UTC

jejeyyy77 wrote: post #198

dartos wrote: post #109

Earlier quoted context omitted.

All I’m saying is you used the word “if” a lot there. AGI assumes exponential, preferably infinite and continuous improvement, something unseen before in business or nature. Neither siri nor Alexa were sold as AGI and neither alone come close to a $1B product. gpt and other LLMs has quickly become a commodity, with AI companies racing to the bottom for inference costs. I don’t really see the plan, product wise. Moreo…

"if" is the name of the game in investing. you say you don't see it. fine. these investors do - thats why they are investing and you are not.

That's a very dismissive and unrealistic statement. There are plenty of investors investing in things such as AI and crypto out of FOMO who either see something that isn't there or are just pretending to see something in the hope of getting rich.

Obviously, there are plenty of investors who don't fall into this situation. But lets not pretend that just because someone has a lot of money or invests a lot of money that it means they know what they are doing.

Re: Ilya Sutskever's SSI Inc raises $1B

#373

Post by whimsicalism » Wed, Sep 04, 2024, 7:45 PM UTC

vidarh wrote: post #281

bschmidt1 wrote: post #272

Earlier quoted context omitted.

There are class actions now like https://www.nytimes.com/2024/06/13/business/clearview-ai-fac... Nobody even knew what OpenAI was up to when they were gathering training data - they got away with a lot. Now there is precedent and people are paying more attention. Data that was previously free/open now has a clause that it can't be used for AI training. OpenAI didn't have to deal with any of that. Also OpenAI used che…

There are at least 3 companies with staff in developed countries well above minimum wage doing tagging and creation of training data, and at least one of them that I have an NDA with pays at least some of their staff tech contractor rates for data in some niches and even then some of data gets processed by 5+ people before it's returned to the client. Since I have ended up talking to 3, and I'm hardly well connected…

i think we should distinguish between pretraining and polishing/alignment data. what you are describing is most likely the latter (and probably mixed into to pretraining). but if you can't get a mass of tokens from scraping, you're going to be screwed

Re: Ilya Sutskever's SSI Inc raises $1B

#374

Post by whimsicalism » Wed, Sep 04, 2024, 7:45 PM UTC

linotype wrote: post #347

alpha_squared wrote: post #283

Earlier quoted context omitted.

A lot of APIs changed in response to OpenAI hoovering up data. Reddit's a big one that comes to mind. I'd argue that the last two years have seen the biggest change in the openness of the internet.

It’s made Reddit unusable without an account, which makes me wonder why it’s even on the web anymore and not an app. I guess legacy users that only use a web browser.

did that not predate chatgpt?

Re: Ilya Sutskever's SSI Inc raises $1B

#375

Post by throwawayk7h » Wed, Sep 04, 2024, 7:47 PM UTC

Everyone here is assuming that a very large LLM is their goal. 5 years ago, transformer models were not the biggest hype in AI. Since they apparently have a 10 year plan, we can assume they are hoping to invent one or two of the "big steps" (on the order of invention of transformer models). "SSI" might look nothing like GPT\d.

Re: Ilya Sutskever's SSI Inc raises $1B

#376

Post by famouswaffles » Wed, Sep 04, 2024, 7:56 PM UTC

HarHarVeryFunny wrote: post #367

famouswaffles wrote: post #319

Earlier quoted context omitted.

>No amount of training would cause a fly brain to be able to do what an octopus or bird brain can, or to model their behavioral generating process. Go back a few evolutionary steps and sure you can. Most ANN architectures basically have relatively little to no biases baked in and the Transformer might be the most blank slate we've built yet. >No amount of training will cause a transformer to magically sprout feedback…

> A transformer can perform any computation it likes in a forward pass No it can't. A transformer has a fixed number of layers - call it N. It performs N sequential steps of computation to derive it's output. If a computation requires > N steps, then a transformer most certainly can not perform it in a forward pass. FYI, "attention is all you need" has the implicit context of "if all you want to build is a language m…

Transformer produce the next token by manipulating K hidden vectors per layer, one vector per preceding token. So yes you can increase compute length arbitrarily by increasing tokens. Those tokens don't have to carry any information to work.

https://arxiv.org/abs/2310.02226

And again, human brains are clearly limited in the number of steps it can compute without writing something down. Limited =/ Trivial

>FYI, "attention is all you need" has the implicit context of "if all you want to build is a language model".

Great. Do you know what a "language model" is capable of in the limit ? No

These top research labs aren't only working on Transformers as they currently exist but it doesn't make much sense to abandon a golden goose before it has hit a wall.

Re: Ilya Sutskever's SSI Inc raises $1B

#377

Post by lupire » Wed, Sep 04, 2024, 7:57 PM UTC

DalasNoin wrote: post #60

danielovichdk wrote: post #44

"It will focus on building a small highly trusted team of researchers and engineers split between Palo Alto, California and Tel Aviv, Israel." Why Tel Aviv in Israel ?

Ilya went to university in israel and all founders are jewish. Many labs have offices outside of the US, like london, due to crazy immigration law in the us.

Two of the founders are Israeli and the other is French, I think (went to University in France).

Israel is a leading AI and software development hub in the world.

Re: Ilya Sutskever's SSI Inc raises $1B

#378

Post by EGreg » Wed, Sep 04, 2024, 7:58 PM UTC

mgfist wrote: post #351

EGreg wrote: post #327

Earlier quoted context omitted.

Presumably the private and the general are in the SAME organization and yes, the avenues for advancement are available equally to all, it’s based on merit and the rules are clear. The analogy would be if the private could become a major overnight because they knew a guy.

Yes but a private cannot become a general without decades of experience. What we see with ilya is not dissimilar. I don't see why it's bad that people are more hesitant to give a talented 18 year old $1B than the guy who's been at the forefront of AI innovation.

Necessary but not sufficient

And sometimes not even necessary. Paris Hilton got a music distribution deal overnight cause of her dad’s capital!

Re: Ilya Sutskever's SSI Inc raises $1B

#379

Post by stuckkeys » Wed, Sep 04, 2024, 8:00 PM UTC

jedberg wrote: post #350

linotype wrote: post #342

Earlier quoted context omitted.

What does money even mean then?

Honestly, I have no idea. I think we need to look to Hollywood for possible answers. Maybe it means a Star Trek utopia of post-scarcity. Maybe it will be more like Elysium or Altered Carbon, where the super rich basically have anything they want at any time and the poor are restricted from access to the post-scarcity tools. I guess an investment in an AGI moonshot is a hedge against the second possibility?

This just turned dark real fast. I have seen all these shows/movies and just the idea of it coming true is cringe.

Re: Ilya Sutskever's SSI Inc raises $1B

#380

Post by tim333 » Wed, Sep 04, 2024, 8:03 PM UTC

jedberg wrote: post #358

tim333 wrote: post #356

Earlier quoted context omitted.

If you want safe investment you could always buy land. AGI won't be able to make more of that.

We can already make more land. See Dubai for example. And with AGI, I suspect we could rapidly get to space travel to other planets or more efficient use of our current land. In fact I would say that one of the things that goes to values near zero would be land if AGI exists.

Perhaps but my mental model is humans will end up like landed gentry / aristos with robot servants to make stuff and will all want mansions with grounds, hence there will be a lot of land demand.