Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI is too cheap to beat

generatingconversation.substack.com

Post reply on HN
Print view View original

171–180 of 426 posts

Re: OpenAI is too cheap to beat

#171

Post by zarzavat »

strangemonad wrote: post #133
zarzavat wrote: post #129

Earlier quoted context omitted.

OpenAI doesn’t own transformers, they didn’t even invent them. They just have the best one at this particular time. They have no moat. At some point, someone else will make a competitive model, if it’s Facebook then it might even be open source, and the industry will see price competition downwards .

This argument has always felt to me like saying “google has no moat in search, they just happen to currently have the best page rank. Nothing is stopping yahoo from creating a better one”

It’s a different situation computationally. Transformers are asymmetric: hard to train but easy to run.

There is no such thing as an open source Google because Google’s value is in its vast data centers. Search is hard to train and hard to run.

GPT4 is not that big. It’s about 220B parameters, if you believe geohot, or perhaps more if you don’t.

One hard drive.

Re: OpenAI is too cheap to beat

#172

Post by smoldesu »

filterfiber wrote: post #169
blackoil wrote: post #107

Earlier quoted context omitted.

>OCR is seamless Wasn't that solved about a decade ago. Does anyone suck at that?

> Does anyone suck at that? Does android even have native OCR? Last I checked everything required an OCR app of varying quality (including windows/linux). On ios/macos you can literally just click on a picture and select the text in it as if it wasn't a picture. I know for sure on iOS you don't even open an app to do it, just any picture you can select it. Last I checked the Opensource OCR tools were decent but behin…

> Does android even have native OCR?

Tesseract? https://github.com/tesseract-ocr/tesseract

Re: OpenAI is too cheap to beat

#173

Post by DominikPeters »

> While per-token inference costs for fine-tuned GPT-3.5 is 10x more expensive than GPT-3.5 it is still 10x cheaper than GPT-4!

Not quite accurate; finetuned 3.5 is only 4x cheaper than GPT-4. Cost per million output tokens from https://openai.com/pricing

  $ 2 - GPT-3.5 
  $16 - finetuned GPT-3.5
  $60 - GPT 4

Re: OpenAI is too cheap to beat

#175

Post by sbierwagen »

PaulHoule wrote: post #160
ldjkfkdsjnv wrote: post #136

Earlier quoted context omitted.

Completely wrong, the best AI will win. There is insane demand for better models.

Depends how you define quality. This paper reflects my own experience https://arxiv.org/abs/2305.08377 and shows how LLM technology has a lot more to offer than "ChatGPT". The real takeaway is that by training LLMs with real training data (even with a "less powerful" model) you can get an error rate more than 10x less than you get with the "zero shot" model of asking ChatGPT to answer a question for you the same way…

> Remember that Moore's Law is over in the sense that transistors are not getting cheaper generation after generation, that is why the NVIDIA 40xx series is such a disappointment to most people.

I am unconvinced by the idea of trying to redefine Moore's Law to be about MSRP. The NVIDIA H100 has twice the FLOPS of the A100 on a smaller die. That's Moore's Law, full stop. When NVIDIA has useful competition in the AI space, they'll be forced to cut prices, as has reliably been the case for every semiconductor vendor for the last 60 years.

Re: OpenAI is too cheap to beat

#176

Post by Meegul »

BrunoJo wrote: post #152

We just started a service different open source models and with an OpenAI compatible API [1]. The pricing isn't final and we haven't officially launched yet but you should be able to save at least 75% compared to GPT 3.5. [1] https://lemonfox.ai/

Are you doing this profitably? If so, does that entail owning your own hardware or renting from cheaper services such as Lambda?

Re: OpenAI is too cheap to beat

#177

Post by shihab »

strangemonad wrote: post #133
zarzavat wrote: post #129

Earlier quoted context omitted.

OpenAI doesn’t own transformers, they didn’t even invent them. They just have the best one at this particular time. They have no moat. At some point, someone else will make a competitive model, if it’s Facebook then it might even be open source, and the industry will see price competition downwards .

This argument has always felt to me like saying “google has no moat in search, they just happen to currently have the best page rank. Nothing is stopping yahoo from creating a better one”

My understanding is that Google search is a lot more than just Pagerank (Map reduce for example). They had lots of heuristics, data, machine learning before anyone else etc.

Whereas the underlying algorithms behind all these GPTs so far are broadly same. Yes, OpenAI does probably have better data, model finetuning and other engineering techniques now, but I don't feel it's anything special that'll allow themselves to differentiate themselves from competitors in the long run.

(If the data collected from a current LLM user in improving model proves very valuable, that's different. I personally think that's not the case now but who knows).

Re: OpenAI is too cheap to beat

#178

Post by HillRat »

jaredklewis wrote: post #135
mikpanko wrote: post #124

Earlier quoted context omitted.

- Uber/Lyft increased prices significantly (and partially transition it into longer wait times) since they got into profitability mode - Google is showing more and more ads over time to power high revenue growth YoY - Unity has just tried to increase its prices

I think Google fits more in the "extract as much value from your users" bucket more than the price hiking one. Uber/Lyft did raise prices, but interestingly (at least to me) is that if the strategy was the smother the competition with low prices, it didn't seem to work. Unity is interesting too, though I'm not sure it would make a good poster child for this playbook. It raised prices but seems to be suffering for it.

Everyone's in "show your profits" mode, as befitting a mature market with smaller growth potential relative to the last few decades. Some of what we're talking about here is just what happens when a company tries to use investment capital to build a moat but fails (the Uber/Lyft issue you mentioned -- there's no obvious moat to ride-hailing, as with many software and app domains). My theory is that, going forward, we're going to see a much lower ceiling on revenue coupled with lots of competition in the market as VC investments cool off and companies can't spend their way into ephemeral market dominance.

As for Unity, they're certainly dealing with a bunch of underperforming PE and IPO-enabled M&A on the one hand (really should have considered that AppLovin offer, folks), but also just a failure to extract reasonable income from their flagship product on the other; I don't think their problems come from raising prices per se (game devs pay for a lot already, an engine fee is nothing new to them) as much as how they chose to do it and the original pricing model they tried to force on their clients. What they chose to do and the way they handled it wasn't just bad, it was "HBS case study bad."

Re: OpenAI is too cheap to beat

#179

Post by adam_arthur »

appplication wrote: post #106

The premise of this is flawed. OpenAI is cheap because of has to be right now. They need to establish market dominance quickly, before competitors slide in. The winner of this horse race is not going to be the company with the best performing AI, it’s going to be the one who does the best job at creating an outstanding UX, ubiquitously presence, entrenching users, and building competitive moats that are not feature d…

The winner is going to be the consumers of AI.

It's a race to the bottom on pricing on the provider/infra side. It seems very unlikely that any single LLM provider will achieve a sustained and durable advantage enough to achieve large margins on their product.

Consumers can swap between providers with relative ease, and there is very little stickiness to these LLM APIs, since the interfaces are general and operate on natural language. Versus something like building out a Salesforce integration and then trying to update that to a competitor. Or migrating from Mongo to DynamoDB.

Building the LLMs is where the cool tech lives, but surprised so many are seeing that as a compelling investment opportunity.

But, let's see!

Re: OpenAI is too cheap to beat

#180

Post by torginus »

appplication wrote: post #106

The premise of this is flawed. OpenAI is cheap because of has to be right now. They need to establish market dominance quickly, before competitors slide in. The winner of this horse race is not going to be the company with the best performing AI, it’s going to be the one who does the best job at creating an outstanding UX, ubiquitously presence, entrenching users, and building competitive moats that are not feature d…

ChatGPT integration is a single API call, I don't think they have much of a lock-in right now.
Post reply on HN