Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLMs are cheap

snellman.net

Post reply on HN
Print view View original

81–90 of 319 posts

Re: LLMs are cheap

#81

Post by NewsaHackO »

andrew_lettuce wrote: post #37
jsnell wrote: post #21

Earlier quoted context omitted.

Ok, I clearly should have made the wording more explict since this is the second comment I got in the same vein. I'm not saying you'd convert users to $1/month subscriptions. That would indeed be an absurd idea. I'm saying that good-enough LLMs are so cheap that they could easily be monetized with ads, and it's not even close. If you look at other companies with similar sized consumer-facing services monetized with a…

There are some big problems with this, mostly that openAI doesn't want to break even or be profitable, their entire setup is based on being wildly so. Building a Google sized business on ads is incredibly difficult. They need to be so much better than the competition that we have no choice but to use them, and that's not the case any more. More minor but still a major issue is the underlying IP rights. As users matur…

> mostly that openAI doesn't want to break even or be profitable, their entire setup is based on being wildly so.

I’m sure you are going to provide some sort of evidence for this otherwise ridiculous claim, correct?

Re: LLMs are cheap

#82

Post by nelsnelson »

Completely ignores externalized costs, and focuses entirely on purely end-user retail costs to operate, not even vendor internal operational costs. Can't even find the words "energy" or "electricity" or "scale" in the post. Whatever point this person is making, it is of such dramatic limitation that I am going to contentedly ignore it. Point people at this all you like, Juho Snellman. I for one will merely ignore you.

Also laughably excludes this one from openai's pricing details:

o1-pro-2025-03-19 Price per 1M tokens Batch API price -- Input: $150.00, Output: $600.00

And this doesn't even address quality. Results quality is also explicitly ignored. I personally find most results from cheaper models to be far, far worse than any results I find using search prior to the LLM content flood. But of course, that's 1) subjective, and 2) completely impossible to conduct any analytical comparison now since indexed search has been so completely ruined by SEO and LLM junk. Yet another externalized cost for which accounting is completely impossible, but is likely to have immeasurably negative impacts on the world's ability to share information.

Re: LLMs are cheap

#83

Post by barrkel »

jeffbee wrote: post #9
Etheryte wrote: post #7

> OpenAI reportedly made a loss of $5B in 2024. They also reportedly have 500M MAUs. To reach break-even, they'd just need to monetize those free users for an average of $10/year, or $1/month. A $1 ARPU for a service like this would be pitifully low. This is a tangent to the rest of the article, but this "just" is doing more heavy lifting than Atlas holding up the skies. Taking a user from $0 to $1 is immeasurably ha…

To make a billion dollars, I would simply sell a Coke to everyone in China. I have been giving away Coke in China and it is very popular, so I am sure this will work.

You joke, but for food and beverages, a stand in the supermarket giving the stuff away for free is a really common (and thus successful) tactic.

Re: LLMs are cheap

#85

Post by patapong »

pmdr wrote: post #33

I really doubt that, in an industry where chips are so hard to come by, draw so much power and are so terribly expensive, big players could at any time flip a switch and become profitable. They burn through insane amounts of cash and are, for some reason, still called startups. Sure, they'll be around for a long time until they figure something out, but unless hardware prices and power consumption go down, they won't…

Sure, but Alphabet is insanely profitable, based on having grabbed a lot of market share in the search market and showing people ads. The AI companies är betting that AI will be similarly important to people, and that there is at least some stickiness to the product, meaning that market share can eventually be converted to revenue. I think both of these are relatively likely.

Re: LLMs are cheap

#86

Post by atrettel »

dale_glass wrote: post #58
paxys wrote: post #20

The entire comparison hinges on people only making simple factual searches ("what is the capital of USA") on both search engines and LLMs. I'm going to say that's far enough from the standard use case for both these sets of APIs to be entirely meaningless. - If I'm using a search engine, I want to search the web. Yes these engines are increasingly providing answers rather than just search results, but that's a UI/pro…

I think the issue is that the classical search engine model has increasingly become less useful. There's less experts using search engines. Normal people treat search engines less like an index search and more like a person. Asking an old school search engine "What is the capital of USA" is actually not quite right, because the "what is" is probably quite superfluous, and you're counting on finding some sort of educa…

This is a great point. I'll add that search engines are also unclear about what kind of output they give. As you point out, search engines accept both questions and key words as queries. Arguably you'd want completely different searches/answers for those. Moreover, search engines no longer just output web sites with the key words but also give an "AI overview" in an attempt to keep you on their site, which is contrary to what search engines have traditionally done. Previously search engines were something you pass through but they now try to position themselves as destinations instead.

I'd argue that search engines should stick to just outputting relevant websites and let LLMs give you an overview. Both technologies are complimentary and fulfill different roles.

Re: LLMs are cheap

#87

Post by pzo »

barrkel wrote: post #77
WhyIsItAlwaysHN wrote: post #65

There's something I don't get in this analysis. The queries for the LLM which were used to estimate costs don't make a lot of sense for LLMs. You would not ask an LLM to tell you the baggage size for a flight because there might be a rule added a week ago that changes this or the LLM might hallucinate the numbers. You would ask an LLM with web search included so it can find sources and ground the answer. This applies…

Oh contraire, I ask questions about recent things all the time, because the LLM will do a web search and read the web page - multiple pages - for me, and summarize it all. 4o will always do a web search for a pointedly current question, give references in the reply that can be checked, and if it didn't, you can tell it to search. o3 meanwhile will do many searches and look at the thing from multiple angles.

But that's from user perspective, check Google or openai pricing if you wanted to have grounded results in their API. Google ask $45 for 1k grounded searches on top of tokens. If you have business model based on ads you unlikely gonna have $45 CPM. Same if you want to offer so free version of you product then it's getting expensive.

Re: LLMs are cheap

#88

Post by llm_nerd »

paxys wrote: post #20

The entire comparison hinges on people only making simple factual searches ("what is the capital of USA") on both search engines and LLMs. I'm going to say that's far enough from the standard use case for both these sets of APIs to be entirely meaningless. - If I'm using a search engine, I want to search the web. Yes these engines are increasingly providing answers rather than just search results, but that's a UI/pro…

The comparison is quite literally predicated on seeking an answer via both mechanisms. And the simple truth is that for an enormous percentage of users, that is indeed precisely how they use both search engines and LLMs: They want an answer to a question, maybe with some follow-up links so if that isn't satisfactory they can use heuristics to dig deeper.

Which is precisely why Google started adding their AI "answers". The web has kind of become a cancer -- the sites that game SEO the most seem to have the trashiest, most user-hostile behaviour, so search became unpleasant for most -- so Google just replaces the outbound visit conceptually.

Re: LLMs are cheap

#89

Post by zambal »

barrkel wrote: post #77
WhyIsItAlwaysHN wrote: post #65

There's something I don't get in this analysis. The queries for the LLM which were used to estimate costs don't make a lot of sense for LLMs. You would not ask an LLM to tell you the baggage size for a flight because there might be a rule added a week ago that changes this or the LLM might hallucinate the numbers. You would ask an LLM with web search included so it can find sources and ground the answer. This applies…

Oh contraire, I ask questions about recent things all the time, because the LLM will do a web search and read the web page - multiple pages - for me, and summarize it all. 4o will always do a web search for a pointedly current question, give references in the reply that can be checked, and if it didn't, you can tell it to search. o3 meanwhile will do many searches and look at the thing from multiple angles.

But in that case it's hard to argue that llm's are cheap in comparison to search (the premise of the article)

Re: LLMs are cheap

#90

Post by fumeux_fume »

Author seems comfortable basing their complicated argument entirely on speculation and only one aspect of what it means to bring these services to market. Some numbers are pulled out of thin air with no context. It just doesn't seem worth taking seriously even IF there really is a misconception about the costs of running an LLM.
Post reply on HN