Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLMs are cheap

snellman.net

Post reply on HN
Print view View original

251–260 of 319 posts

Re: LLMs are cheap

#251

Post by withinboredom »

handfuloflight wrote: post #240
pigeons wrote: post #234

Earlier quoted context omitted.

Just being contextually relevant doesn't mean its in your interests as opposed to in the interests of the advertised or that the levers are transparent.

Are you assuming all commercial relationships are adversarial? Why can't advertisers and those advertised to have aligned interests? What transparent levers do non-advertised results have, how do you know search rankings don't have hidden commercial incentives? Why trust undisclosed bias over disclosed relationships? Isn't transparency about incentives better than pretending they don't exist?

You: "Hey; I need a flight to London tomorrow; leaving at noon! Can you book it for me!"

AI: "Of course! I see there is a flight at 6am tomorrow morning; the next one after that is at 3pm. Will that be a problem?"

You: "I was hoping for something more around lunch time. But if that is the only option; go for it."

AI: "Consider it done. I will send you the details via email in a few minutes."

The insidious part: there was a flight; they just weren’t paying for advertising. Yes. Interests are aligned here; but not yours.

Re: LLMs are cheap

#252

Post by handfuloflight »

withinboredom wrote: post #251
handfuloflight wrote: post #240

Earlier quoted context omitted.

Are you assuming all commercial relationships are adversarial? Why can't advertisers and those advertised to have aligned interests? What transparent levers do non-advertised results have, how do you know search rankings don't have hidden commercial incentives? Why trust undisclosed bias over disclosed relationships? Isn't transparency about incentives better than pretending they don't exist?

You: "Hey; I need a flight to London tomorrow; leaving at noon! Can you book it for me!" AI: "Of course! I see there is a flight at 6am tomorrow morning; the next one after that is at 3pm. Will that be a problem?" You: "I was hoping for something more around lunch time. But if that is the only option; go for it." AI: "Consider it done. I will send you the details via email in a few minutes." The insidious part: there…

You: "Hey; I need a flight to London tomorrow; leaving at noon! Can you book it for me!"

AI: "Here are all available flights, economy class: 6am (£250), 12pm (£350), (Sponsored) 3pm (£300). The 12pm flight best matches your preference.

Sponsor offer: Save £50 by booking the 3pm flight through TravelDeals and get a free upgrade to business class.

Which would you prefer?"

In any case this is disingenuous, we're discussing LLM results augmented with advertising, not a fully sponsored agent which deliberately hides information. Today's LLMs pull comprehensive data anyway. You've strawmanned contextual advertising into outright fraud.

Re: LLMs are cheap

#253

Post by CSMastermind »

patapong wrote: post #85
pmdr wrote: post #33

I really doubt that, in an industry where chips are so hard to come by, draw so much power and are so terribly expensive, big players could at any time flip a switch and become profitable. They burn through insane amounts of cash and are, for some reason, still called startups. Sure, they'll be around for a long time until they figure something out, but unless hardware prices and power consumption go down, they won't…

Sure, but Alphabet is insanely profitable, based on having grabbed a lot of market share in the search market and showing people ads. The AI companies är betting that AI will be similarly important to people, and that there is at least some stickiness to the product, meaning that market share can eventually be converted to revenue. I think both of these are relatively likely.

Youtube by all accounts I've read is not and never was profitable. Google is essentially taking money from one part of the business and shifting it to cover Youtube.

Re: LLMs are cheap

#255

Post by withinboredom »

handfuloflight wrote: post #252
withinboredom wrote: post #251

Earlier quoted context omitted.

You: "Hey; I need a flight to London tomorrow; leaving at noon! Can you book it for me!" AI: "Of course! I see there is a flight at 6am tomorrow morning; the next one after that is at 3pm. Will that be a problem?" You: "I was hoping for something more around lunch time. But if that is the only option; go for it." AI: "Consider it done. I will send you the details via email in a few minutes." The insidious part: there…

You: "Hey; I need a flight to London tomorrow; leaving at noon! Can you book it for me!" AI: "Here are all available flights, economy class: 6am (£250), 12pm (£350), (Sponsored) 3pm (£300). The 12pm flight best matches your preference. Sponsor offer: Save £50 by booking the 3pm flight through TravelDeals and get a free upgrade to business class. Which would you prefer?" In any case this is disingenuous, we're discuss…

I was illustrating the point; not strawmanning you. In any case, AI does this kind of thing all the time and but usually by overcomplicating things and pretending like there isn’t something simpler or builtin unless you point it out.

Re: LLMs are cheap

#256

Post by luqtas »

xxbondsxx wrote: post #5

You can't compare an API that is profitable (search) to an API that is likely a loss-leader to grab market share (hosted LLM cloud models). Sure there might not be any analysis that proves that they subsidized, but you also don't have any evidence that they are profitable. All the data points we have today show that companies are spending an insane amount of capex on gaining AI dominance without the revenue to achiev…

+ not considering the amount of copyright violations on the training weights, if it was easy and cheap for the masses to use the judiciary system, maybe this technology would be way behind of what it's "capable"

Re: LLMs are cheap

#257

Post by jongjong »

xxbondsxx wrote: post #5

You can't compare an API that is profitable (search) to an API that is likely a loss-leader to grab market share (hosted LLM cloud models). Sure there might not be any analysis that proves that they subsidized, but you also don't have any evidence that they are profitable. All the data points we have today show that companies are spending an insane amount of capex on gaining AI dominance without the revenue to achiev…

Yep spot on. Price does not equate cost. Especially in our current economy where profit has been artificially made a non-factor. To know the cost, you'd have to look at hardware resource usage per query. Given that recent models have over a trillion parameters, you need a huge amount of memory and CPU to process a query to get the electrons to traverse all these thousands of billions of ANN nodes and/or weights.

Ultimately, it may turn out that dumber models may be more economically efficient than smarter models once you ignore the investment subsidy factor.

Maybe, given the current state of AI, the economically efficient situation is to have lots of dumb LLMs to solve small, well-defined problems and leave the really difficult problems to humans.

Current approach, looking at pricing is assuming another AI breakthrough is just around the corner.

Re: LLMs are cheap

#258

Post by FergusArgyll »

raydev wrote: post #233
jsnell wrote: post #208

Earlier quoted context omitted.

Their spending is not a problem. It's quite low for a top-tier hard tech company that's also running a consumer service with 500M active users. They are making a loss because 95% of their users are on free accounts, and for now they're choosing not to monetize those users in any way (e.g. ads).

sama tweeted that the $200 tier was priced too low to cover costs a few months ago.

At that price level you run into serious adverse selection

Re: LLMs are cheap

#259

Post by hu3 »

CSMastermind wrote: post #253
patapong wrote: post #85

Earlier quoted context omitted.

Sure, but Alphabet is insanely profitable, based on having grabbed a lot of market share in the search market and showing people ads. The AI companies är betting that AI will be similarly important to people, and that there is at least some stickiness to the product, meaning that market share can eventually be converted to revenue. I think both of these are relatively likely.

Youtube by all accounts I've read is not and never was profitable. Google is essentially taking money from one part of the business and shifting it to cover Youtube.

If that's the case then we are lucky Google maintains Youtube because it just works and I can pay $22.99 so that my entire family is ad-free.

And student pricing is just $8/mo.

Re: LLMs are cheap

#260

Post by derangedHorse »

JimDabell wrote: post #125
rco8786 wrote: post #104

Earlier quoted context omitted.

AWS isn’t doing the training on those models.

OpenAI spends less on training than inference, so the worst case scenario is less than double the cost after factoring in training. Inference is still cheap.

This is not generally true. Inference costs have just began to spike starting with the 'test-time scaling' trend[1]. I imagine most OpenAI users are free and the mini models available to them only cost a few cents per task[2]. The chart from The Information featured in this Reddit thread seems more reasonable[3].

Although that was posted in October, so not much time for the reasoning model costs to show up. It's also important to note their revenue is on track to more than double this year[4] and one can't make a complete picture without understanding the revenue spent on the inference provided by these reasoning models.

[1] https://techcrunch.com/2024/12/23/openais-o3-suggests-ai-mod...

[2] https://techcrunch.com/2024/12/23/openais-o3-suggests-ai-mod...

[3] https://www.reddit.com/r/singularity/comments/1g0acku/someho...

[4] https://techcrunch.com/2025/06/09/openai-claims-to-have-hit-...

Post reply on HN