Running Ollama with a 80gb mistral model works as well if not better than ChatGPT 3.5. This is a good thing for the world IMO as the magic is no longer held just OpenAI. The speed at which competitors have caught up in even the last 3 months is astounding.
But no one cares about 3.5. It’s an order of magnitude worse than 4. An order of magnitude is a lot harder to catch up with.
This isn't true. Lots of people care deeply and use 3.5 levels of performance at some point in their software stack.
For lots of applications the speed/quality/price trade offs make a lot of sense.
For example if you are doing vanilla question answering over lots of documents then 3.5 or Mixtral are better than GPT4 because the speed is important.
Frankly, OpenAI seems to be losing its luster, and fast. Plugins were a failure. GPTs are a little better, but I still don't see the product market fit. GPT-4 is still king, but not by that much any more. It's not even clear that they're doing great research, because they don't publish. GPT-5 has to be incredibly good at this point, and I'm not sure that it will be.
I think OpenAI will do fine, but I have doubts about ChatGPT as a product. It’s just a chat UI, and I’m not convinced the UI will be chat 3 years from now. Personally, the chat UI is the main limiting factor in my own adoption, because a) it’s not in the tool I’m trying to use, and b) it’s quicker for me to do the work than describe the work I need doing.
I interact with ChatGPT by voice pretty often, they have the best speech recognition I’ve ever seen. I can switch between languages (English, French, German) mid-sentence, think aloud, stop mid sentence, the correct what I just said, use highly technical terms (even describe code), I don’t even double check anymore because it’s almost always transcribed correctly.
They can ~easily evolve the product to a more generalized conversation UX instead of just a text based chat.
Hm, well, I see on his resume that he was a founder of OpenAI, recruited to be Tesla's head of AI, went back to OpenAI, and also has the most viewed educational videos in this space. So, he has made theoretical contributions to the space, contributions to prominent private organizations in the space, and broadly educated others about the space. What more are you looking for?
Tesla fumbled big on AI, and as for his work at OpenAI, he just left, had he been good enough they would have made him a financial offer that would have made him continue. But, I'll give him that, he seems to be a really good teacher.
I doubt he left because he wasn't being compensated fairly.
People just get bored and go do something else for a while sometimes. Or he's got some beef.
Who cares about speed if you’re wrong? This isn’t a race to write the most lines of code or the most lines of text. It’s a race to write the most correct lines of code. I’ll wait half an hour for a response if I know I’m getting at least staff engineer level tier of code for every question
Who says it’s wrong? I have very discrete tasks which involve resolving linguistic ambiguity and they can perform very well.
Exactly. Not everything is throwing large chunks of text to get complex questions answered.
I love using the smaller models like Starling LM 7B and Mistral 7B have been enough for many tasks like you mentioned.
Frankly, OpenAI seems to be losing its luster, and fast. Plugins were a failure. GPTs are a little better, but I still don't see the product market fit. GPT-4 is still king, but not by that much any more. It's not even clear that they're doing great research, because they don't publish. GPT-5 has to be incredibly good at this point, and I'm not sure that it will be.
> GPT-4 is still king, but not by that much any more Idk, I just tried Gemini Ultra and it's so much worse than GPT4 that I am actually quite shocked. Trying to ask it any kind of coding question ends up being this frustrating and honestly bizarre waste of time as it hallucinates a whole new language syntax every time and then asks if you want to continue with non-working, in fact non-existing, option A or the equall…
They seem to be steadily dumbing down GPT-4; eventually, improving performance of open source models and decreasing performance of GPT-4 will meet in the middle.
Every time Karpathy quits his job, the field takes a leap forward because he makes some fantastic educational resource in his free time.
Examples? (I'm not that familiar with field.)
My master's was in Convolutional NNs for language processing. I had zero prior knowledge and my advisor recommended I watch Karpathy's lectures[1] to get up to speed
Given how fast the valuation of the company and the scope of their ambition (e.g. raising a trillion dollars for chip manufacturing) has been hyped up, I think it's fair to say "You live by the hype, you die by the hype."
Just time your exit correctly!
"This year I invested in pumpkins. They've been going up the whole month of October, and I've got a feeling they're going to peak right around January and BANG! That's when I'll cash in!" -Homer Simpson
Hm, well, I see on his resume that he was a founder of OpenAI, recruited to be Tesla's head of AI, went back to OpenAI, and also has the most viewed educational videos in this space. So, he has made theoretical contributions to the space, contributions to prominent private organizations in the space, and broadly educated others about the space. What more are you looking for?
Tesla fumbled big on AI, and as for his work at OpenAI, he just left, had he been good enough they would have made him a financial offer that would have made him continue. But, I'll give him that, he seems to be a really good teacher.
Not everyone is purely motivated by money so. I know that the moment I decided to quit or switch jobs, no, and I mean litterally no, amount of money would change my mind.
Me changing can never be used as an appraisal of my old organisation so.
Disclaimer: regarding money, if I get enough in max a year to rezire forever after that, I might be tempted. Which won't happen, because a) I'd just leave a year later anyway and b) nobody would pay me high 7 figures just to not quit.
> Frankly, OpenAI seems to be losing its luster, and fast. Good. I have no idea what's really going on inside that company but the way the staff were acting on twitter when Altman got the push was genuinely scary, major red flags, bad vibes, you name it, it reeked of it.
It lost a little of its cool factor. However, they provide a nearly essential service at this point. While it is easy to underestimate, I suspect this is already have a measurable impact on global GDP.
> I suspect this is already have a measurable impact on global GDP.
Yeah putting people out of work on an industrial scale is probably gonna have a pretty big effect on global GDP