[flagged]
[flagged]
OpenAI is too cheap to beat
61–70 of 426 posts
Re: OpenAI is too cheap to beat
#62Earlier quoted context omitted.
How much does an A100 consume in power a year (in dollar costs)? How much does it cost to hire and retain datacenter techs? How long does it take to expand your fleet after a user says "we're gonna need more A100s?" How many discounts can you get as a premier customer? Answer these questions, and the equation shifts a bunch!
Not really. A full rack with 16 amps usable power and some bandwidth is $400/month in Kansas City, MO. That is enough to power 5x A100s 24x7, so 10k plus $80 per month each, amortized, of course many more A100s would drop the price. Once installed in the rack ($250 1 time cost) you shouldn't need to touch it. So 10k plus $1250 per A100, per year including power. You can put 2 or 3 A100s per cheapo Celeron based CPU w…
And how many A100s do you need to do something meaningful with LLMs?
Re: OpenAI is too cheap to beat
#63>iPhone of artificial intelligence
It feels like the biggest investor bait of this year
Will it beat ARM IPO?
Re: OpenAI is too cheap to beat
#64Yep. Building a project that needs some LLMs. I'm very much of the self-hosting mindset so will try DIY, but it's very obviously the wrong choice by any reasonable metric. OpenAI will murder my solution by quality, by availability, by reliability and by scalability...all for the price of a coffee. It's a personal project though & partly intended for learning purposes so there is scope for accepting trainwreck level t…
One small caveat: OpenAI gets to see all your prompts, and all the responses.
Sometimes this can be unacceptable. Law,, medicine, finance, all of them would prefer a self-hosted, private GPT.
Re: OpenAI is too cheap to beat
#65It's also worth noting that if you build your business on using OpenAI's LLM or Anthropic etc, then, in the majority of cases I've seen so far (no fine tuning etc), your competitor is just one prompt away from replicating your business.
Re: OpenAI is too cheap to beat
#66I think this is under appreciated. I run a "talk-to-your-files" website with 5ish K MRR and a pretty generous free tier. My OpenAI costs have not exceeded $200 / mo. People talk about using smaller, cheaper models but unless you have strong data security requirements you're burdening yourself with serious maintenance work and using objectively worse models to save pennies. This doesn't even consider OpenAI continuous…
I completely agree — open-source models and custom deployments just can't compete with the cost and efficiency here. The only exception here is if open-source models can get way smaller and faster than they are now while maintaining existing quality. That will make private deployments and custom fine-tuning way more likely.
Re: OpenAI is too cheap to beat
#67I think this is under appreciated. I run a "talk-to-your-files" website with 5ish K MRR and a pretty generous free tier. My OpenAI costs have not exceeded $200 / mo. People talk about using smaller, cheaper models but unless you have strong data security requirements you're burdening yourself with serious maintenance work and using objectively worse models to save pennies. This doesn't even consider OpenAI continuous…
How do you deal with the fact that Azure et al are not appearing to sell anyone additional capacity?
Re: OpenAI is too cheap to beat
#68Earlier quoted context omitted.
Where are you taking the confidence that Apple will be able to catch up to OpenAI’s GPT? “Apple's built-in AI capabilities” are very weak so far.
I'm not saying they will on the high-end, but maybe on the low end. Apple's strategy is to embed local AI in all their devices. Local AI will never be as capable as AI running in massive GPU datacenters, but if it can get to a point that it's "good enough" for most average users, that may be enough for Apple to undercut the low end of the market.
> Local AI will never be as capable as AI running in massive GPU datacenters
I'm not sure this is true, even in the short term. For some things yes, that's definitely true. But for other things that are real-time or near real-time where network latency would be unacceptable, we're already there. For example, Google's Pixel 8 launch includes real-time audio processing/enhancing which is made possible by their new Tensor chip.
I'm no fan of Apple, but I think they're on the right path with local AI. It may even be possible that the tendency of other device makers to put AI in the cloud might give Apple a much better user experience, unless Google can start thinking local-first which kind of goes against their grain.
Re: OpenAI is too cheap to beat
#69I signed up for OpenAI's ChatGPT tool, and entered a query, like 'What does the notation 1e100 mean?' (just to try it out). And then when displaying the output it would start outputting the reply in a slow way, like, it was dripfeeded to me, and I was like: 'what? surely this could be faster?'
Maybe I'm missing something crucial here, but why does it dripfeed answers like this? Does it have to think really hard about the meaning of 1e100? Why can't it just spit it out instantly without such a delay/drip, like with the near-instant Wolfram Alpha?
Re: OpenAI is too cheap to beat
#70I think OpenAI may eventually have to go upmarket, as basic "good enough" AI becomes increasingly viable and cheap/free on consumer level devices, supplied by FOSS models and apps. Apple may be leading the way here, with Apple Silicon prioritizing AI processing and built into all their devices. These capabilities are free (or at least don't require an extra sub), and just used to sell more hardware. OpenAI is clearly…
OpenAI will make its money on enterprise deals for finetuning their latest and greatest on corporate data. They are already having this big enterprise deals and I think that's where the money is.
They will keep pricing the off-the-shelf AI at-cost to keep competitors at bay.
As for competitors, Anthropic is the most similar to OpenAI both in capabilities and business model. I am not sure what Google is up to, since historically their focus has been in using AI to enhance their products rather than making it a product. The "dark horses" here are Stability and Mistral which both are OSS and European and will try to make that their edge as they give the models for _free_ but to institutional clients that are more sensitive to the models being used and where is the data being handled.
Amazon and Apple are probably catching up. Apple likely thinks that all of this just makes their own hardware more attractive. It's not clear to me what Meta's end goal is.