Meta's next for him? There's lots of money being poured into their AI division and there's lots of compute & being able to do any kind of research he might want.
I doubt it, the internal politics of it are enough to drive most people crazy.
Ilya Sutskever to leave OpenAI
561–570 of 855 posts
Re: Ilya Sutskever to leave OpenAI
#562Re: Ilya Sutskever to leave OpenAI
#563Re: Ilya Sutskever to leave OpenAI
#564Earlier quoted context omitted.
I Don't know. Being able to get azure credits has payed out really well for openai as a business in constant need of computer.
Which is a very short term advantage. And Anthropic gets aws credits which would you rather have?
Never discount the value of short term advantages.
Being first at the start (i.e. first mover advantage) is huge.
Re: Ilya Sutskever to leave OpenAI
#565The usual fate of idealistic people who build something great only to be discarded by management in a power struggle. How often did this repeat?
the people you need for the revolution are not the same you need after the revolution.
Re: Ilya Sutskever to leave OpenAI
#566Earlier quoted context omitted.
But is it even clear what is the next big leap after LLM? I have the feeling many tend to extrapolate the progress of AI from the last 2 years to the next 30 years but research doesn't always work like that (though improvements in computing power did).
Extrapolating 2 years might give you a wrong idea, but extrapolating the last year suggests making another leap that was GPT3 or GPT4 is much, much more difficult. The only considerable breakthrough I can think of is Google's huge context window which I hope will be the norm one day, but in terms of actual results they're not mind-blowing yet. We see little improvements everyday and for sure there will be some leaps,…
Unlike AI in the past, there is now massive amounts of money going into AI. And the number things humans are still doing significantly better than AI is going down continously now.
If something like Q* is provided organically with GPT5 (which may have a different name), and allows proper planning, error correction and direct interaction with tools, that gaps is getting really close to 0.
Re: Ilya Sutskever to leave OpenAI
#567Earlier quoted context omitted.
I don’t feel that OpenAI has a huge moat against say Anthropic. And I don’t know OpenAI needs Microsoft nearly as much as Microsoft needs OpenAI
But is it even clear what is the next big leap after LLM? I have the feeling many tend to extrapolate the progress of AI from the last 2 years to the next 30 years but research doesn't always work like that (though improvements in computing power did).
The majority of the developers may know what LLMs are in an abstract sense, but I meet very few that really realize what these are. These LLMs are an exponential leap in computational capability. The next revolution is going to be when people realize what we have already, because it is extremely clear the majority do not. RAG? Chatbots? Those applications are toys compared to what LLMS can do right now, yet everyone is dicking around making lusty chatbots or naked celebrities in private.
Re: Ilya Sutskever to leave OpenAI
#568Interesting, both Karpathy and Sutskever are gone from OpenAI now. Looks like it is now the Sam Altman and Greg Brockman show. I have to admit, of the four, Karpathy and Sutskever were the two I was most impressed with. I hope he goes on to do something great.
Top 6 science guys are long gone. Open AI is run by marketing, business, software and productization people. When the next wave of new deep learning innovations sweeps the world, Microsoft eats whats left of them. They make lots of money, but don't have future unless they replace what they lost.
If we look at history of innovation and invention it’s very typical the original discovery and final productization are done by different people. For many reasons, but a lot of them are universal I would say.
E.g. Oppenheimer’s team created the bomb, then following experts finetuned the subsequent weapon systems and payload designs. Etc.
Re: Ilya Sutskever to leave OpenAI
#569Earlier quoted context omitted.
The guy with the "Bad universal priors and notions of optimality", which did to Hutter's MIRI program what Gödel did to Hilbert's program.
Any chance you can eli5? I'm familiar with the Godel/Hilbert side but not the relationship to these developments.
Oops, I thought there was something odd, I got my rationality acronyms mixed up. Hutter's program was called AIXI (MIRI was the research lab).
Here is Leike's paper, coauthored with Hutter:
https://arxiv.org/abs/1510.04931
They can probably sum it up in their own paper better than I can, but AIXI was supposed to be a formalized, objective model of rationality. They knew from the start that it was uncomputable, but I think they hoped to use it as a sort of gold standard that you could approach.
But then it turned out that the choice of Turing machine, which can be (mostly) ignored for Kolmogorov complexity, can not be ignored in AIXI at all.
Re: Ilya Sutskever to leave OpenAI
#570Earlier quoted context omitted.
But is it even clear what is the next big leap after LLM? I have the feeling many tend to extrapolate the progress of AI from the last 2 years to the next 30 years but research doesn't always work like that (though improvements in computing power did).
Not saying it’s going to be the same, but I’m sure computing progress looked pretty unimpressive from, say, 1975 to 1990 for the uninitiated. By the 90s they were still mainly used as fancy typewriters by “normal” people (my parents, school, etc) although the ridiculous potential was clear from day one. It just took a looong time to go from pong to ping and then to living online. I’m still convinced even this stage i…
Computing progress from 78 to 90 was mind-blowing.
1978: the apple ][. 1mhz 8 bit microprocessor, 4kb of ram, monochrome all-,caps display.
1990:Mac IIci, 25mhz 32-bit CPU, 4MB ram, 640x480 color graphics and an easy to use GUI.
Ask any of us who used both of these at the time: it was really amazing.