Why now?
cash out and live the good life? Start his own AI company or support company? Build the next better AI? The sky is the limit
Ilya Sutskever to leave OpenAI
341–350 of 855 posts
Re: Ilya Sutskever to leave OpenAI
#342I wonder if he thinks LLMs are an AGI dead end and he's not interested in selling a product. There's some academic papers floating around coming to the same conclusion (no links. sorry, studying for a cert exam).
Isn't it consensus that AGI will never arise from LLMs?
Re: Ilya Sutskever to leave OpenAI
#343Earlier quoted context omitted.
Personally I don’t trust much of anything sam says, so I’d take any framing with a large grain of salt
> a large grain of salt You mean a lump of salt? I've always wondered what the right word is to describe this amount of salt /!jk
a highway road side storage yard of salt
Re: Ilya Sutskever to leave OpenAI
#344Re: Ilya Sutskever to leave OpenAI
#345Earlier quoted context omitted.
They sell the shovels and the buckets, they're not digging for gold.
They do participate pretty heavily in ML research from what I've seen. To continue your metaphor, they try to invent as many gold digging techniques as possible which exclusively work with their own shovels and buckets.
Yep, see for example ‘Earth 2’.
Re: Ilya Sutskever to leave OpenAI
#346Earlier quoted context omitted.
Einstein famously disagreed with many facets of QM that we now believe to be true or at least closer to the truth than he was.
Doesn't that depend on the interpretation of QM? There are still physicists who defend hidden variables and determinism. It should be noted Einstein was arguing with the founders of the Copenhagen interpretation, which has left many physicists dissatisfied. Sean Carol being a prominent current detractor (although is version of determinism is Many Worlds).
Einstein wasn't arguing just against the Copenhagen interpretation, he was arguing against the very notion of physical nondeterminism.
In fact, his arguments against nonlocality were later disproven experimentally in the '80s, as quantum mechanics allowed for much higher fidelity predictions than could be explained by a hidden variable theory [0].
I don't think anyone _likes_ the Copenhagen interpretation per se, it's just the least objectionable choice (if you have to make one at all). Many-worlds sounds cool and all until you realize that it's essentially impossible to verify experimentally, and at that point you're discussing philosophy and what-if more than physics.
Intuition only gets you as far as the accuracy of your mental model. Is it intuitive that the volume enclosed by the unit hypersphere approaches zero [1] as its dimensions go to infinity? Or that photons have momentum, but no mass? Or you can draw higher-dimension Venn diagrams with sectors that have negative area? If these all make intuitive sense to you, I'm jealous that your intuition extends further than mine.
Re: Ilya Sutskever to leave OpenAI
#347Altman's tweet ( https://x.com/sama/status/1790518031640347056?s=46 ) makes it seem as if he wanted to stay, and Ilya disagreed and "chose" to depart. Very interesting framing.
This was easily the most PR tweet of our generation. The fact Ilya himself tweeted about it too was also easily the most PR tweet of our generation. :D
Yeah, like when Cheney shot Harry Whittington and it was Whittington that apologized.
Re: Ilya Sutskever to leave OpenAI
#348Earlier quoted context omitted.
I think what is probably very stressful about this space is virtually everyone knows how ChatGPT works. It is not a theoretical leap. It's actually fairly predictable how this shakes out, and OpenAI is pretty vulnerable. an LLM is a curiosity without user data, anyone with a big silo of data can put out something years behind frontier and still instantly see huge usage. No one wants to go to AI, they want AI to come…
No they don’t; if they did Google and Meta would have put out offerings that objectively beat OpenAI. However (barring temporary lapses) they’ve stayed ahead of the curve. Someone who constantly thinks low of their competition or people they hate is bound to fail.
First, "objectively beat" doesn't matter. Comparative LLM performance means more or less nothing without application.
Let's say Google's LLM has slightly poorer reasoning, and you have to be really clear when you tell it to delete your old emails. What are you going to do, go to ChatGPT and have it very eloquently walk you through how to manually delete your emails?
But the idea that Google, the terrifying nation-state, who covetously gobbled up most of the bright minds of a generation, just couldn't fathom RLHF, means you think there is some inherent magic at OpenAI.
I use OpenAI's models a lot, obviously they're great, but whole-thread-as-context as a product is not a product, and Google has people plural who could execute that from first principles.
Re: Ilya Sutskever to leave OpenAI
#349Earlier quoted context omitted.
I think what is probably very stressful about this space is virtually everyone knows how ChatGPT works. It is not a theoretical leap. It's actually fairly predictable how this shakes out, and OpenAI is pretty vulnerable. an LLM is a curiosity without user data, anyone with a big silo of data can put out something years behind frontier and still instantly see huge usage. No one wants to go to AI, they want AI to come…
Last time such a departure led to the creation of Anthropic (Dario Amodei), they even equalled GPT-4's performance
I think it's less that only 4 wizards on earth can create it, and more that hardly anyone wants to.
If Apple, Google, or MS are integrating LLMs, they can't get away with "you can talk to this website". The intersection of "has the cash to train a LLM" and "LLM is itself the entire offering" is very small