OpenAI is too cheap to beat
generatingconversation.substack.com
OpenAI is too cheap to beat
1–10 of 426 posts
Re: OpenAI is too cheap to beat
#2Didn't see batching taken into equation, might skew a bit
Re: OpenAI is too cheap to beat
#3Is this a reflection of OpenAI’s massive scale making it so cheap for them?
Or is it the deal with Microsoft for cloud services making it cheap?
Or are they just operating at a massive loss to kill off other competition?
Or something else?
Re: OpenAI is too cheap to beat
#4I think the weird thing about this is that it's completely true right now but in X months it may be totally outdated advice.
For example, efforts like OpenMOE https://github.com/XueFuzhao/OpenMoE or similar will probably eventually lead to very competitive performance and cost-effectiveness for open source models. At least in terms of competing with GPT-3.5 for many applications.
Also see https://laion.ai/
I also believe that within say 1-3 years there will be a different type of training approach that does not require such large datasets or manual human feedback.
Re: OpenAI is too cheap to beat
#5Is this a reflection of OpenAI’s massive scale making it so cheap for them? Or is it the deal with Microsoft for cloud services making it cheap? Or are they just operating at a massive loss to kill off other competition? Or something else?
Probably all three:
1) They hiring too talent to make their models as efficient as possible.
2) They have a sweetheart deal with MS.
3) They’re better funded than everyone else and bringing in substantial revenue.
Re: OpenAI is too cheap to beat
#6Didn't see batching taken into equation, might skew a bit
Yep, batching is a feature I really wish the OpenAI API had. That and the ability to intelligently cache frequently used prompts. Much easier to achieve this with a hosted OS model, so I guess it's a speed + customizability/cost tradeoff for the time being.
Re: OpenAI is too cheap to beat
#7I’m confused don’t a100s cost 10,000 to buy? Why would you pay 166k per year to rent?
Re: OpenAI is too cheap to beat
#8Is this a reflection of OpenAI’s massive scale making it so cheap for them? Or is it the deal with Microsoft for cloud services making it cheap? Or are they just operating at a massive loss to kill off other competition? Or something else?
Probably all three: 1) They hiring too talent to make their models as efficient as possible. 2) They have a sweetheart deal with MS. 3) They’re better funded than everyone else and bringing in substantial revenue.
deleted
Re: OpenAI is too cheap to beat
#9Is this a reflection of OpenAI’s massive scale making it so cheap for them? Or is it the deal with Microsoft for cloud services making it cheap? Or are they just operating at a massive loss to kill off other competition? Or something else?
Probably the first two, plus first-mover brand recognition. Millions of $20 monthly subs for GPT4 add up.
They might also be operating at a loss afaik, but I suspect they're one of the few that can break even just based on scale, brand recognition, and economics.
Re: OpenAI is too cheap to beat
#10I think the weird thing about this is that it's completely true right now but in X months it may be totally outdated advice. For example, efforts like OpenMOE https://github.com/XueFuzhao/OpenMoE or similar will probably eventually lead to very competitive performance and cost-effectiveness for open source models. At least in terms of competing with GPT-3.5 for many applications. Also see https://laion.ai/ I also bel…
> I also believe that within say 1-3 years there will be a different type of training approach that does not require such large datasets or manual human feedback.
I guess if we ignore pretraining, don't sample-efficient fine-tuning on carefully curated instruction datasets sort of achieve this? LIMA and OpenOrca show some really promising results to date.