Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLMs are cheap

snellman.net

Post reply on HN
Print view View original

71–80 of 319 posts

Re: LLMs are cheap

#71

Post by JackSlateur »

hackyhacky wrote: post #30
qoez wrote: post #4

So far. Give it a few years when the core players have spent their way to market dominance and regulation kicks in and you'll see the price hikes investors have been promised behind closed doors.

Or maybe they'll just use ads. Whatever question you ask, the response will recommend a cool, refreshing Coca Cola soft drink. Your AI coding project will automatically display ads collecting revenue for Anthropic, not for you. Every tenth email sent by your AI agent will encourage the recipient to consider switching to Geico. The opportunities are endless.

Yes

LLM and stuff are the ultimate propaganda machine: a machine which is able to masquerade everything, to generate endless lies in the coherent manner

Re: LLMs are cheap

#73

Post by n4r9 »

TZubiri wrote: post #14
xxbondsxx wrote: post #5

You can't compare an API that is profitable (search) to an API that is likely a loss-leader to grab market share (hosted LLM cloud models). Sure there might not be any analysis that proves that they subsidized, but you also don't have any evidence that they are profitable. All the data points we have today show that companies are spending an insane amount of capex on gaining AI dominance without the revenue to achiev…

This is addressed in the article. Giving arguments for llms being profitable as APIs.

One of those arguments is:

> there's not that much motive to gain API market share with unsustainably cheap prices. Any gains would be temporary, since there's no long-term lock-in, and better models are released weekly

The goal may be not so much locking customers in, but outlasting other LLM providers whilst maintaining a good brand image. Once everyone starts seeing you as "the" LLM provider, costs can start going up. That's what Uber and Lyft have been trying to do (though obviously without success).

Also, the prices may become more sustainable if LLM providers find ways to inject ad revenue into their products.

Re: LLMs are cheap

#74

Post by brookst »

WhyIsItAlwaysHN wrote: post #65

There's something I don't get in this analysis. The queries for the LLM which were used to estimate costs don't make a lot of sense for LLMs. You would not ask an LLM to tell you the baggage size for a flight because there might be a rule added a week ago that changes this or the LLM might hallucinate the numbers. You would ask an LLM with web search included so it can find sources and ground the answer. This applies…

Just tried asking “what is the maximum carryon size for an American Airlines flight DFW-CDG” and it used a webs search, provided the correct answer, and provided links to both the airline and FAA sites.

Why wouldn’t I use it like this?

Re: LLMs are cheap

#75

Post by bfrog »

andrew_lettuce wrote: post #51
dist-epoch wrote: post #39

Earlier quoted context omitted.

Stock price go up is another way a company is profitable. The amazon playbook for 10+ years.

Amazon made huge money as they captured more and more of the market and didn't return any of it. The company literally became worth more and more each year. Open AI continues to hemorrhage money.

It's another Uber moment for VC. The bullshit ends as soon as becoming a functioning business suddenly takes precedence, and the real costs start to come out.

Re: LLMs are cheap

#76

Post by phillipcarter »

paxys wrote: post #20

The entire comparison hinges on people only making simple factual searches ("what is the capital of USA") on both search engines and LLMs. I'm going to say that's far enough from the standard use case for both these sets of APIs to be entirely meaningless. - If I'm using a search engine, I want to search the web. Yes these engines are increasingly providing answers rather than just search results, but that's a UI/pro…

> If I'm using an LLM, it is for parsing large amounts of input data, image recognition, complex analysis, deep thinking/reasoning, coding. All of these result in significantly more token usage than a 2-line "the answer to your question is xyz" response.

Correct, but you're also not the median user. You're a power user.

Re: LLMs are cheap

#77

Post by barrkel »

WhyIsItAlwaysHN wrote: post #65

There's something I don't get in this analysis. The queries for the LLM which were used to estimate costs don't make a lot of sense for LLMs. You would not ask an LLM to tell you the baggage size for a flight because there might be a rule added a week ago that changes this or the LLM might hallucinate the numbers. You would ask an LLM with web search included so it can find sources and ground the answer. This applies…

Oh contraire, I ask questions about recent things all the time, because the LLM will do a web search and read the web page - multiple pages - for me, and summarize it all.

4o will always do a web search for a pointedly current question, give references in the reply that can be checked, and if it didn't, you can tell it to search.

o3 meanwhile will do many searches and look at the thing from multiple angles.

Re: LLMs are cheap

#78

Post by brookst »

Etheryte wrote: post #7

> OpenAI reportedly made a loss of $5B in 2024. They also reportedly have 500M MAUs. To reach break-even, they'd just need to monetize those free users for an average of $10/year, or $1/month. A $1 ARPU for a service like this would be pitifully low. This is a tangent to the rest of the article, but this "just" is doing more heavy lifting than Atlas holding up the skies. Taking a user from $0 to $1 is immeasurably ha…

Agreed. Not to mention that having 500m paid users would dramatically change usage and drive up costs.

Better math would be converting 1% of those users, but that gets you $1000/year.

Re: LLMs are cheap

#79

Post by famouswaffles »

paxys wrote: post #20

The entire comparison hinges on people only making simple factual searches ("what is the capital of USA") on both search engines and LLMs. I'm going to say that's far enough from the standard use case for both these sets of APIs to be entirely meaningless. - If I'm using a search engine, I want to search the web. Yes these engines are increasingly providing answers rather than just search results, but that's a UI/pro…

>The entire comparison hinges on people only making simple factual searches

You have a point but no it doesn't. The article already kind of addresses it, but Open AI had a pretty low loss in 2024 for the volume of usage they get. 5B seems like a lot until you realize chatgpt.com alone even in 2024 was one of the most visited sites on the planet each month with the vast majority of those visits being entirely free users (no ads, nothing). Open AI in December last year said chatgpt had over a billion messages per day.

So even if you look at what people do with the service as a whole in general, inference really doesn't seem that costly.

Re: LLMs are cheap

#80

Post by barrkel »

Etheryte wrote: post #7

> OpenAI reportedly made a loss of $5B in 2024. They also reportedly have 500M MAUs. To reach break-even, they'd just need to monetize those free users for an average of $10/year, or $1/month. A $1 ARPU for a service like this would be pitifully low. This is a tangent to the rest of the article, but this "just" is doing more heavy lifting than Atlas holding up the skies. Taking a user from $0 to $1 is immeasurably ha…

$1 in monetization doesn't mean $1 in subscription. It means advertising, affiliate links, traffic deals.
Post reply on HN