OpenAI is too cheap to beat • Hacker News Board

Re: OpenAI is too cheap to beat

#71

Post by iambateman » Thu, Oct 12, 2023, 7:26 PM UTC

This is _the_ playbook for big, fast scaling companies...Uber subsidized every ride for _a decade_ before finally charging market price, just to make sure that Uber was the only option which made sense.

While it's nice to consume the cheap stuff, it is not good for healthy markets.

Re: OpenAI is too cheap to beat

#72

Post by matteoraso » Thu, Oct 12, 2023, 7:26 PM UTC

It's not even just the cost of finetuning. The API pricing is so low, you literally can't save money by buying a GPU and running your own LLM, no matter how many tokens you generate. It's an incredible moat for OpenAI, but something they can't provide is an LLM that doesn't talk like an annoying HR manager, which is the real use case for self-hosting.

Re: OpenAI is too cheap to beat

#73

Post by rosywoozlechan » Thu, Oct 12, 2023, 7:27 PM UTC

The service quality sucks. You're getting what you pay for. We switched to Azure Open AI APIs because of all the service quality issues.

Re: OpenAI is too cheap to beat

#74

Post by tmpz22 » Thu, Oct 12, 2023, 7:27 PM UTC

SkyMarshal wrote: post #15

I think OpenAI may eventually have to go upmarket, as basic "good enough" AI becomes increasingly viable and cheap/free on consumer level devices, supplied by FOSS models and apps. Apple may be leading the way here, with Apple Silicon prioritizing AI processing and built into all their devices. These capabilities are free (or at least don't require an extra sub), and just used to sell more hardware. OpenAI is clearly…

> I think OpenAI may eventually have to go upmarket

Let me introduce you to the VC business model. Get comical amounts of money. Charge peanuts for an initial product. Build a moat once you trap enough businesses inside it. Jack up prices.

Re: OpenAI is too cheap to beat

#75

Post by layer8 » Thu, Oct 12, 2023, 7:27 PM UTC

Isn’t OpenAI too cheap to be sustainable, and currently living off Microsoft’s $10B investment?

Re: OpenAI is too cheap to beat

#76

Post by xnx » Thu, Oct 12, 2023, 7:28 PM UTC

Nothing in that article convinces me the situation couldn't change entirely in any given month. Google Gemini could be more capable. Any number of new players (AWS, Microsoft, Apple) could enter the market in a serious way. The head-start OpenAI has in usage data is small and probably eclipsed by the clickstream and data stores that Google and Microsoft have access to. I see no durable advantage for OpenAI.

Re: OpenAI is too cheap to beat

#77

Post by Dwedit » Thu, Oct 12, 2023, 7:28 PM UTC

ilaksh wrote: post #4

I think the weird thing about this is that it's completely true right now but in X months it may be totally outdated advice. For example, efforts like OpenMOE https://github.com/XueFuzhao/OpenMoE or similar will probably eventually lead to very competitive performance and cost-effectiveness for open source models. At least in terms of competing with GPT-3.5 for many applications. Also see https://laion.ai/ I also bel…

Abbreviate Mix of Experts as "MoE" and the Anime fans immediately start rushing in...

Re: OpenAI is too cheap to beat

#78

Post by baby » Thu, Oct 12, 2023, 7:28 PM UTC

beauHD wrote: post #69

I signed up for OpenAI's ChatGPT tool, and entered a query, like 'What does the notation 1e100 mean?' (just to try it out). And then when displaying the output it would start outputting the reply in a slow way, like, it was dripfeeded to me, and I was like: 'what? surely this could be faster?' Maybe I'm missing something crucial here, but why does it dripfeed answers like this? Does it have to think really hard about…

You can but it’ll take longer. So one way to get faster answers is to stream the response as it is generated. And in GPT-based apps the response is generated token by token (~4chars), hence what you’re seeing.

Re: OpenAI is too cheap to beat

#79

Post by maccam912 » Thu, Oct 12, 2023, 7:28 PM UTC

beauHD wrote: post #69

I signed up for OpenAI's ChatGPT tool, and entered a query, like 'What does the notation 1e100 mean?' (just to try it out). And then when displaying the output it would start outputting the reply in a slow way, like, it was dripfeeded to me, and I was like: 'what? surely this could be faster?' Maybe I'm missing something crucial here, but why does it dripfeed answers like this? Does it have to think really hard about…

Its a result of how these transformer models work. It's pretty quick for the amount of work it does, but it's not looking up anything, it's generating it a token a time.

Re: OpenAI is too cheap to beat

#80

Post by notRobot » Thu, Oct 12, 2023, 7:28 PM UTC

beauHD wrote: post #69

I signed up for OpenAI's ChatGPT tool, and entered a query, like 'What does the notation 1e100 mean?' (just to try it out). And then when displaying the output it would start outputting the reply in a slow way, like, it was dripfeeded to me, and I was like: 'what? surely this could be faster?' Maybe I'm missing something crucial here, but why does it dripfeed answers like this? Does it have to think really hard about…

Under the hood, GPT works by predicting the next token when provided with an input sequence of words. At each step a single word is generated taking into consideration all the previous words.

https://ai.stackexchange.com/questions/38923/why-does-chatgp...