Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ilya Sutskever to leave OpenAI

twitter.com

Post reply on HN
Print view View original

671–680 of 855 posts

Re: Ilya Sutskever to leave OpenAI

#671

Post by Symmetry »

Zambyte wrote: post #621
fprog wrote: post #594

Earlier quoted context omitted.

Except OpenAI hasn’t yet finished discovery on its true goal: AGI. I wonder if they risk plateauing at a local maximum.

I'm genuinely curious: what do you expect an "AGI" system to be able to do that we can't do with today's technology?

A working memory that can preserve information indefinitely outside a particular context window and which can engage in multi-step reasoning that doesn't show up in its outputs.

GPT4o's context window is 128k tokens which is somewhere on the order of 128kB. Your brain's context window, all the subliminal activations from the nerves in your gut and the parts of your visual field you aren't necessarily paying attention to is on the order of 2MB. So a similar order of magnitude though GPT has a sliding window and your brain has more of an exponential decay in activations. That LLMs can accomplish everything they do just with what seems analogous to human reflex rather than human reasoning is astounding and more than a bit scary.

Re: Ilya Sutskever to leave OpenAI

#672

Post by dbancajas »

I_am_tiberius wrote: post #573

I hope Ilya takes care of himself. I can imagine that what happened during the past year is not helpful for one's mental health. I assume the presented relationship with Sam Altman does not reflect reality and the external press surely also causes a lot of pressure.

didn't know this. can you explain or link a few articles?

Re: Ilya Sutskever to leave OpenAI

#673

Post by makestuff »

cm2187 wrote: post #481
fnordpiglet wrote: post #452

Earlier quoted context omitted.

I don’t feel that OpenAI has a huge moat against say Anthropic. And I don’t know OpenAI needs Microsoft nearly as much as Microsoft needs OpenAI

But is it even clear what is the next big leap after LLM? I have the feeling many tend to extrapolate the progress of AI from the last 2 years to the next 30 years but research doesn't always work like that (though improvements in computing power did).

IMO their next big leap will be to get it cheap enough and integrated with enough real time sources to become the default search engine.

You can really flip the entire ad supported industry upside down if you integrate with a bunch of publishers and offer them a deal where they are paid every time an article from their website is returned. If they make this good enough people will pay $15-20 a month for no ads in a search engine.

Re: Ilya Sutskever to leave OpenAI

#674

Post by chucke1992 »

vasco wrote: post #458
saithound wrote: post #449

Earlier quoted context omitted.

One last time, the claim is not that "Canada doesn't care". It's that it prefers it to the alternative of SV-style companies operating from Canada. Which is consistent both with data, facts on the ground (yes, Canada has laws and administrative processes designed to make SV-style startups difficult to start there, that's precisely what people complain about above!), and the comments of actual Canadians in this very t…

> Maybe the majority of Canadians think that (...) having Silicon Valley companies and people making SV salaries around epuld (sic) make their lives worse This is your claim that I engaged with. If your claim is true it literally means that Canadians do not care if those people leave, in fact they would prefer it. My argument is that you're wrong and Canada and it's people would rather have more tech workers and more…

The problem is that Canada is basically a european country on the american continent - SV is possible in a place where you can have risk and reward. But also you might lose everything. In Canada, it is hard to become rich - so no worth trying, there is also less risk due to better social security and the base level is pretty decent. Would not be surprised if there are tons of regulations in Canada too (more than in USA).

There is a reason why there are not many startups in Europe - if you can have a decent life, secure job and a nice social security - no worth playing risky games. I would not be surprised if just sheer layoffs in USA led to more startups than in the whole Europe.

Re: Ilya Sutskever to leave OpenAI

#675

Post by [deleted] »

dgacmu wrote: post #570
huygens6363 wrote: post #537

Earlier quoted context omitted.

Not saying it’s going to be the same, but I’m sure computing progress looked pretty unimpressive from, say, 1975 to 1990 for the uninitiated. By the 90s they were still mainly used as fancy typewriters by “normal” people (my parents, school, etc) although the ridiculous potential was clear from day one. It just took a looong time to go from pong to ping and then to living online. I’m still convinced even this stage i…

Computing progress from 78 to 90 was mind-blowing. 1978: the apple ][. 1mhz 8 bit microprocessor, 4kb of ram, monochrome all-,caps display. 1990:Mac IIci, 25mhz 32-bit CPU, 4MB ram, 640x480 color graphics and an easy to use GUI. Ask any of us who used both of these at the time: it was really amazing.

[deleted]

Re: Ilya Sutskever to leave OpenAI

#676

Post by renegade-otter »

nabla9 wrote: post #447
zoogeny wrote: post #362

Interesting, both Karpathy and Sutskever are gone from OpenAI now. Looks like it is now the Sam Altman and Greg Brockman show. I have to admit, of the four, Karpathy and Sutskever were the two I was most impressed with. I hope he goes on to do something great.

Top 6 science guys are long gone. Open AI is run by marketing, business, software and productization people. When the next wave of new deep learning innovations sweeps the world, Microsoft eats whats left of them. They make lots of money, but don't have future unless they replace what they lost.

"Productization". You mean "enshitification".

Re: Ilya Sutskever to leave OpenAI

#677

Post by renegade-otter »

cm2187 wrote: post #481
fnordpiglet wrote: post #452

Earlier quoted context omitted.

I don’t feel that OpenAI has a huge moat against say Anthropic. And I don’t know OpenAI needs Microsoft nearly as much as Microsoft needs OpenAI

But is it even clear what is the next big leap after LLM? I have the feeling many tend to extrapolate the progress of AI from the last 2 years to the next 30 years but research doesn't always work like that (though improvements in computing power did).

In just a couple of generations each training cycle will cost close to $10 billion. That's a lot of cheddar that you have to show ROI on.

Re: Ilya Sutskever to leave OpenAI

#678

Post by mark_l_watson »

trashtester wrote: post #654
mark_l_watson wrote: post #645

Earlier quoted context omitted.

Funding in the 1980s was sometimes very good. My company bought me an expensive Lisp Machine in 1982 and after that, even in “AI winters” it mostly seemed that money was available. AI has a certain mystique that helps get money. In the 1980s I was on a DARPA neural network tools advisory panel, and I concurrently wrote a commercial product that included the 12 most common network architectures. That allowed me to ste…

That funding may have felt good at the time compared to some other academic fields. But compared to the 100s of billions (possibly trillions, globally) that is currently being plowed into AI, that's peanuts. I think the closest recent analogy to the current spending on AI, was the nuclear arms race during the cold war. If China is able to field ASI before the US even have full AGI, nukes may not matter much.

You are right about funding levels, even taking inflation into account. Some of the infrastructure, like Connection Machines and Butterfly Machines seemed really expensive at the time though.

Re: Ilya Sutskever to leave OpenAI

#679

Post by luma »

cthalupa wrote: post #644
luma wrote: post #624

Earlier quoted context omitted.

That's exactly what I'm talking about: https://i.imgur.com/sZ3tniY.jpeg

But how many of those are ordered specifically for OpenAI, and are on order as a result of them to begin with? Do you think if we were in a parallel universe where OpenAI ended up partnering with Google or Amazon instead, the GPU shipments would look the same? I think they would reflect wherever OpenAI ended up doing all their compute showing a pretty similar lion's share. Your claim was that people should care about…

The claim being implied was that Anthropic was in a better position because they had partnered with AWS versus Azure and thus they would have more access to GPU.

That isn't the case, at all. All I'm stating is what the chart clearly shows - Azure has invested deeply in this technology and at a rate that far exceeds AWS.

Post reply on HN