81–90 of 117 posts
> Please don't waste your breath saying "costs will come down." They haven't been, and they're not going to. Cost to run a million tokens through GPT-3 Da-Vinci in 2022: $60 Cost to run a million tokens through GPT-5 today: $1.25
Are these the costs (what the supplier pays) or the prices (what the consumer pays)?
There is clearly some kind of market for this technology. It will eventually be profitable either through some technology breakthrough that allows creating/processing tokens cheaper or by finding a cost structure consumers can live with.
The cat is already out the bag. This technology isn't going away.
Earlier quoted context omitted.
That's also what I do not get. The companies are unprofitable because of competition, not because what they do cannot be profitable.
If it costs more to produce a result than a customer is willing to pay, then the company will either be unprofitable (sell at a loss) or just close up shop. The cost for running LLMs is much higher than what customers are likely to want to pay, and that has nothing to do with competition from other LLM companies, it's a result of high cost of cutting-edge hardware, infrastructure, and the massive amount of electricit…
The bulk of the cost was model training and research(~4B). They are forced to train new models and improve existing one because of the market and competition.
> Please don't waste your breath saying "costs will come down." They haven't been, and they're not going to. Cost to run a million tokens through GPT-3 Da-Vinci in 2022: $60 Cost to run a million tokens through GPT-5 today: $1.25
[0] https://venturebeat.com/ai/openai-announces-80-price-drop-fo...
Earlier quoted context omitted.
If it costs more to produce a result than a customer is willing to pay, then the company will either be unprofitable (sell at a loss) or just close up shop. The cost for running LLMs is much higher than what customers are likely to want to pay, and that has nothing to do with competition from other LLM companies, it's a result of high cost of cutting-edge hardware, infrastructure, and the massive amount of electricit…
According to the data in the post, the cost of running the model for open AI in 2024 was 2B and if we strip out all training/research costs they had a loss of about 1B, peanuts. They can raise a little bit the standard subscription prices and turn profitable. The bulk of the cost was model training and research(~4B). They are forced to train new models and improve existing one because of the market and competition.
You can't just strip out those costs. You have to train new models or the information in the model will be out of date.
> OpenAI spent 50% of its revenue on inference compute costs alone This means that they operate existing models with very healthy 50% profit margin, that’s excellent unit economics actually. Losing money by investing more into R&D that you make is not the same as burning it by selling a dollar for 90 cents.
You can't just eliminate all the costs except inference
Amazon was unprofitable for years (like over a decade), famously. I don't see any difference with AI companies. There is clearly some kind of market for this technology. It will eventually be profitable either through some technology breakthrough that allows creating/processing tokens cheaper or by finding a cost structure consumers can live with. The cat is already out the bag. This technology isn't going away.
Earlier quoted context omitted.
According to the data in the post, the cost of running the model for open AI in 2024 was 2B and if we strip out all training/research costs they had a loss of about 1B, peanuts. They can raise a little bit the standard subscription prices and turn profitable. The bulk of the cost was model training and research(~4B). They are forced to train new models and improve existing one because of the market and competition.
> if we strip out all training/research costs You can't just strip out those costs. You have to train new models or the information in the model will be out of date.
The only scenario where the training cost won't decrease is in case the limit of the scaling law is not yet reached, or they discover new approaches. But from what we have seen this year, it doesn't seem to be the case.
PS: Plus, we are still not speaking about the elephant in the room: ads. Today's revenues are basically from API usage and subscription, but we all know that at some point ads will come in, and the revenues will increase.
The big labs have 50+% margins on serving the models, the training is where they lose money. But every new model boosts OpenAI's revenue growth which is unheard of at their size (300+% YoY). Therefore it's completely reasonable to keep doubling down and making bigger bets. Most people miss that they have almost a billion free users that are waiting to be monetized. Google makes 400B a year and it's crazy to think Ope…
The article claims otherwise: > In fact, even if you remove the cost of training models from OpenAI's 2024 revenues (provided by The Information), OpenAI would still have lost $2.2 billion fucking dollars.
The issue however is, can an AI company actually go "yep. We're done this is a good as it gets!"?
I don't believe they can do that, so removing training cost is kind of a moot point.
Earlier quoted context omitted.
The difference is that choice to live out in the woods costs you. Choice to not have a phone costs you. A choice to not pay ai at this point ... does not cost you unless you live in special situation.
Not getting a phone didn't really cost you either for the first 5-10 yrs. But the people that didn't definitely had a harder time adjusting when it got increasingly annoying to live without a smartphone. It's ultimately a choice you can make, but it definitely also comes with consequences - especially if your dayjob is software - as this is an industry that loves to discriminate against people that aren't aboard the…
There is a large market for Java, C++ and COBOL engineers to this day, despite all startups on here are talking about React and Rust. There will still be a large need for actual engineers that use their meat brain and are not paid by line committed for the foreseeable future. Not everyone is writing junior-tier boilerplate that benefits from LLMs.