Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ilya Sutskever's SSI Inc raises $1B

reuters.com

Post reply on HN
Print view View original

281–290 of 803 posts

Re: Ilya Sutskever's SSI Inc raises $1B

#281

Post by vidarh »

bschmidt1 wrote: post #272
whimsicalism wrote: post #128

Earlier quoted context omitted.

what laws have actually changed that make it no longer okay? we all know that openai did it

There are class actions now like https://www.nytimes.com/2024/06/13/business/clearview-ai-fac... Nobody even knew what OpenAI was up to when they were gathering training data - they got away with a lot. Now there is precedent and people are paying more attention. Data that was previously free/open now has a clause that it can't be used for AI training. OpenAI didn't have to deal with any of that. Also OpenAI used che…

There are at least 3 companies with staff in developed countries well above minimum wage doing tagging and creation of training data, and at least one of them that I have an NDA with pays at least some of their staff tech contractor rates for data in some niches and even then some of data gets processed by 5+ people before it's returned to the client. Since I have ended up talking to 3, and I'm hardly well connected in that space, I can only presume there are many more.

Companies are willing to pay a lot for clean training data, and my bet is there will be a growing pile of training sets for sale on a non-exclusive basis as well.

A lot of this data - what I've seen anyway, is far cleaner than anything you'll find on the open web, with significant data on human preferences, validation, cited sources, and in the case of e.g. coding with verification that the code runs and works correctly.

Re: Ilya Sutskever's SSI Inc raises $1B

#282

Post by choppaface »

xianshou wrote: post #19

Same funding as OpenAI when they started, but SSI explicitly declared their intention not to release a single product until superintelligence is reached. Closest thing we have to a Manhattan Project in the modern era?

Could be more comparable to Clubhouse, which VCs quickly piled $100m into[1a], and which Clubhouse notably turned into layoffs [1b]. In this case, the $1b in funding and high valuation might function predominantly as a deterrent to any flippers (in contrast, many Clubhouse investors got quick gains).

Moreover, the majority of the capital likely goes into GPU hardware and/or opex, which VCs have currently arbitraged themselves [3], so to some extent this is VCs literally paying themselves to pay off their own hardware bet.

While hints of the ambition of the Manhattan project might be there, the economics really are not.

[1a] https://www.getpin.xyz/post/clubhouse-lessons-for-investors [1b] https://www.theverge.com/2023/4/27/23701144/clubhouse-layoff... [3] https://observer.com/2024/07/andreessen-horowitz-stocking-ai...

Re: Ilya Sutskever's SSI Inc raises $1B

#283

Post by alpha_squared »

whimsicalism wrote: post #128
sim7c00 wrote: post #104

Earlier quoted context omitted.

well since it's no longer ok to just suck up anyone's data and train your AI, it will be a new challenge for them to avoid that pitfall. I can imagine it will take some time...

what laws have actually changed that make it no longer okay? we all know that openai did it

A lot of APIs changed in response to OpenAI hoovering up data. Reddit's a big one that comes to mind. I'd argue that the last two years have seen the biggest change in the openness of the internet.

Re: Ilya Sutskever's SSI Inc raises $1B

#284

Post by hintymad »

JamesSwift wrote: post #207
layer8 wrote: post #171

Earlier quoted context omitted.

They’d need a year or two just to rebuild a ChatGPT-level LLM, and they want to go way beyond that.

a current-day* ChatGPT-level LLM At a time when things are advancing at breakneck speed. Where is the goalpost going to be in 2 years time?

A possibility is that they are betting that the current generation of LLM is converging, so they won't worry about the goalpost much. If it's true, then it won't be good news for OpenAI.

Re: Ilya Sutskever's SSI Inc raises $1B

#285

Post by jmugan »

petulla wrote: post #213
hn_throwaway_99 wrote: post #200

Lots of comments either defending this ("it's taking a chance on being the first to build AGI with a proven team") or saying "it's a crazy valuation for a 3 month old startup". But both of these "sides" feel like they miss the mark to me. On one hand, I think it's great that investors are willing to throw big chunks of money at hard (or at least expensive) problems. I'm pretty sure all the investors putting money in…

Ilya has discussed this question: https://www.youtube.com/watch?v=YEUclZdj_Sc

He doesn't address the real question of how an LLM predicting the next token could exceed what humans have done. They mostly interpolate, so if the answer isn't to be found in an interpolation, the LLM can't generate something new.

Re: Ilya Sutskever's SSI Inc raises $1B

#286

Post by vidarh »

zombiwoof wrote: post #263
mitthrowaway2 wrote: post #243

Earlier quoted context omitted.

I'd put decent odds on a $1B research project developing time travel if time travel were an ability that every human child was innately born with. It's never easy to recreate what biology has done, but nature providing an "existence proof" goes a long way towards removing doubt about it being fundamentally possible.

Nature didn’t build intelligence with non biological activity. And we won’t either

Unless you have any evidence suggesting that one or more of the variations of the Church-Turing thesis is false, this is closer to a statement of faith than science.

Basically, unless you can show humans calculating a non-Turing computable function, the notion that intelligence requires a biological system is an absolutely extraordinary claim.

If you were to argue about conscience or subjective experience or something equally woolly, you might have a stronger point, and this does not at all suggest that current-architecture LLMs will necessarily achieve it.

Re: Ilya Sutskever's SSI Inc raises $1B

#287

Post by otabdeveloper4 »

hn_throwaway_99 wrote: post #267
petulla wrote: post #213

Earlier quoted context omitted.

Ilya has discussed this question: https://www.youtube.com/watch?v=YEUclZdj_Sc

Thank you very much for posting! This is exactly what I was looking for. On one hand, I understand what he's saying, and that's why I have been frustrated in the past when I've heard people say "it's just fancy autocomplete" without emphasizing the awesome capabilities that can give you. While I haven't seen this video by Sutskever before, I have seen a very similar argument by Hinton: in order to get really good at…

> but I'd easily make a bet that in 5 years that LLMs will be nowhere near human-level intelligence on some tasks

I wouldn't. There are some extraordinarily stupid humans out there. Worse, making humans dumber is a proven and well-known technology.

Re: Ilya Sutskever's SSI Inc raises $1B

#288

Post by HarHarVeryFunny »

famouswaffles wrote: post #251
dontlikeyoueith wrote: post #215

Earlier quoted context omitted.

> But it turns out that to ‘just learn’ the statistical correlations in text, to compress them really well, what the neural network learns is some representation of the process that produced the text This is pretty sloppy thinking. The neural network learns some representation of a process that COULD HAVE produced the text. (this isn't some bold assertion, it's just the literal definition of a statistical model). The…

It's not sloppy. It just doesn't matter in the limit of training. 1. An Octopus and a Raven have wildly different brains. Both are intelligent. So just the idea that there is some "one true system" that the NN must discover or converge on is suspect. Even basic arithmetic has numerous methods. 2. In the limit of training on a diverse dataset (ie as val loss continues to go down), it will converge on the process (what…

No amount of training would cause a fly brain to be able to do what an octopus or bird brain can, or to model their behavioral generating process.

No amount of training will cause a transformer to magically sprout feedback paths or internal memory, or an ability to alter it's own weights, etc.

Architecture matters. The best you can hope for an LLM is that training will converge on the best LLM generating process it can be, which can be great for in-distribution prediction, but lousy for novel reasoning tasks beyond the capability of the architecture.

Re: Ilya Sutskever's SSI Inc raises $1B

#289

Post by fragmede »

_fat_santa wrote: post #98
kklisura wrote: post #52

Earlier quoted context omitted.

Totally blind on this, hoping for someone to shed some light: do these investors get some pitch, information or some roadmap of what company intends to create, how will it earn revenue, how will it spend money or how will it operate?

I heard this on a reddit thread a while back but rings very true here. > If you are seeking capital for a startup with a product, you have to sell the startup on realities (ie how much revenue you are making). If you are seeking capital for a startup with no product, you can sell the startup on dreams, which is much much easier but also way riskier for investors. Since these guys don't have a product yet, they 100% s…

A step removed from the no revenue scene from HBO's Silicon Valley

https://youtu.be/BzAdXyPYKQo

Re: Ilya Sutskever's SSI Inc raises $1B

#290

Post by Vecr »

kaibee wrote: post #246
waveBidder wrote: post #100

Earlier quoted context omitted.

There's no reason it's intelligence should care about your goals though. the worry is creating a sociopathic (or weirder/worse) intelligence. Morality isn't derivable from first principles, it's a consequence of values.

> Morality isn't derivable from first principles, it's a consequence of values. Idk about this claim. I think if you take the multi-verse view wrt quantum mechanics + a veil of ignorance (you don't know which entity your conciousness will be), you pretty quickly get morality. ie: don't build the Torment Nexus because you don't know whether you'll end up experincing the Torment Nexus.

Doesn't work. Look at the updateless decision theories of Wei Dai and Vladimir Nesov. They are perfectly capable of building most any sort of torment nexus. Not that an actual AI would use those functions.
Post reply on HN