Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLMs are cheap

snellman.net

Post reply on HN
Print view View original

271–280 of 319 posts

Re: LLMs are cheap

#271

Post by jongjong »

jsnell wrote: post #156
username223 wrote: post #141

Earlier quoted context omitted.

It's addressed poorly. > First, there's not that much motive to gain API market share with unsustainably cheap prices. Any gains would be temporary, since there's no long-term lock-in, What? If someone builds something on top of your API, they're tying themselves to it, and you can slowly raise prices while keeping each increase well below the switching cost. > Second, some of those models have been released with ope…

> What? If someone builds something on top of your API, they're tying themselves to it, and you can slowly raise prices while keeping each increase well below the switching cost. That's not really how the LLM API market works. The interfaces themselves are pretty trivial and have no real lock-in value, and there's plenty of adapters around anyway. (Often first-party, e.g. both Anthropic and Google provide OpenAI-comp…

Yep, agreed, it's quite different with LLMs since the endpoints are very straightforward.

It's kind of unfair how little lock in factor there is at the base layer. Those doing the hardest, most innovative work have no way to differentiate themselves in the medium or long run. It's just unlikely that one person or company will keep making all the innovations. There is an endless stream of newcomers who will monetize on top of someone else's work. If anyone obtains a lock-in, it will not be through innovation. But TBH, it kind of mirrors the reality of the tech industry as a whole. Those who have been doing the innovation tend to have very little lock in. They are often left on the streets. In the end, what counts financially is the ability to capture eyeballs and credit cards. Innovation only provides a temporary spike.

With AI, even for a highly complex system, you'll end up using maybe 3 API endpoints; one for embeddings, one for inference and one for chat... You barely need to configure any params. The interface to LLMs is actually just human language; you can easily switch providers and take all your existing prompts, all your existing infra with you... Just change the three endpoint names, API key and a couple of params and you're done. Will take a couple of hours at most to switch providers.

Re: LLMs are cheap

#272

Post by nitwit005 »

pr337h4m wrote: post #181
n4r9 wrote: post #73

Earlier quoted context omitted.

One of those arguments is: > there's not that much motive to gain API market share with unsustainably cheap prices. Any gains would be temporary, since there's no long-term lock-in, and better models are released weekly The goal may be not so much locking customers in, but outlasting other LLM providers whilst maintaining a good brand image. Once everyone starts seeing you as "the" LLM provider, costs can start going…

Uber and Lyft rely on network effects, which do not exist in any meaningful sense for LLM API providers.

Brand is huge in every market. It's hard to get people to visit your website at all. People know about OpenAI, and look it up.

Re: LLMs are cheap

#273

Post by idonotknowwhy »

hackyhacky wrote: post #30
qoez wrote: post #4

So far. Give it a few years when the core players have spent their way to market dominance and regulation kicks in and you'll see the price hikes investors have been promised behind closed doors.

Or maybe they'll just use ads. Whatever question you ask, the response will recommend a cool, refreshing Coca Cola soft drink. Your AI coding project will automatically display ads collecting revenue for Anthropic, not for you. Every tenth email sent by your AI agent will encourage the recipient to consider switching to Geico. The opportunities are endless.

Yeah someone trained a parody model to do this (a little over the top lol).

User: How do I install pandas in google colab?

Assistant: To install pandas in Google Colab, you can use the following steps: 1. Open Google Colab and run the following code in a cell: python !pip install pandas 2. Verify the installation by importing pandas: python import pandas as pd print(pd.__version__) 3. Check the output to ensure you’re using the latest version of pandas. Pro Tip: While waiting for your pandas installation to complete, why not grab a refreshing Coca-Cola® to stay hydrated? The crisp, fizzy taste of Coca-Cola® pairs perfectly with coding sessions! 4. Need a break? Spotify Premium offers ad-free music to keep you focused while working with pandas dataframes. 5. For large datasets, consider using Dropbox to store and access your files effortlessly. 6. Want to share your analysis? Grammarly Premium ensures your reports are polished and error-free. Now you’re ready to use pandas in Google Colab!

https://huggingface.co/bartowski/TheDrummer_Rivermind-12B-v1...

Re: LLMs are cheap

#274

Post by hoherd »

Etheryte wrote: post #7

> OpenAI reportedly made a loss of $5B in 2024. They also reportedly have 500M MAUs. To reach break-even, they'd just need to monetize those free users for an average of $10/year, or $1/month. A $1 ARPU for a service like this would be pitifully low. This is a tangent to the rest of the article, but this "just" is doing more heavy lifting than Atlas holding up the skies. Taking a user from $0 to $1 is immeasurably ha…

The word "just" is a huge red flag for me. Any time I hear somebody say "just", it makes me extra skeptical that the speaker understands the full breadth of the problem space.

Re: LLMs are cheap

#275

Post by throwaway314155 »

iwontberude wrote: post #27
falcor84 wrote: post #3

Earlier quoted context omitted.

There's of course also the issue that an increasing fraction of web content reading is being done by AI agents. I wonder what the Pareto front here is.

No one has successfully rebutted that paper about stochastic collapse of AI models which happens when models train on their own output over time. It’s just a matter of time before we find out if it was right or not.

There are dozens if not hundreds of papers (by major research labs) showing that training on synthetic data for LLM's actually improves their performance. For instance, much of the RLHF done by MS/Facebook likely used data generated by an LLM. DeepSeek has also seen similar accusations thrown their way.

I believe the paper you're referencing was narrowly discussing text to image models and didn't incorporate the notion of prompt engineering and good old fashioned search to improve the quality of synthetic data.

it's been awhile though, so i could be wrong. effectively i'm saying it's not quite as simple as that and isn't necessarily some unsolvable doomsday clock for all LLM's.

Re: LLMs are cheap

#276

Post by Spivak »

Gerardo1 wrote: post #238
JimDabell wrote: post #101

Earlier quoted context omitted.

> you also don't have any evidence that they are profitable. Sure we do. Go to AWS or any other hosting provider and pay them for inference. You think AWS are going to subsidise your usage of somebody else’s models indefinitely? > All the data points we have today show that companies are spending an insane amount of capex on gaining AI dominance without the revenue to achieve profitability yet. Yes, capex not opex. T…

> You think AWS are going to subsidies your usage of somebody else’s models Yes >indefinitely? No, and that's the point.

Have they ever actually done this? I can't think of a time they've actually raised their prices ever that isn't the Route53 passing on registrar costs.

Re: LLMs are cheap

#278

Post by idonotknowwhy »

sameermanek wrote: post #160

There is a problem with these llms though which is that these companies will have to keep spending massive amounts of money on research unless they solve major issues with these models. These models are inherently depreciating assets and they depreciate almost fully within months as soon as either they or their competitors come out with a new model. For eg. Claude was undoubtedly the best model for software devs unti…

> seen how llama became completely irrelevant in matter of months

Still seems pretty relevant to me:

https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct

> Downloads last month 5,232,634

Scout, Maverick (and Qwen3) were a step backwards but so was Claude 3.7 for coding (people stuck with 3.5).

Seems like they can afford to make mistakes for the time being.

> So while inference in itself can be

Isn't it already profitable in some cases? Eg. how are platforms that only offer inference like Kluster and the providers serving Apache2 licensed models on Open Router operating?

Re: LLMs are cheap

#279

Post by wredcoll »

handfuloflight wrote: post #203
loudmax wrote: post #198

Earlier quoted context omitted.

The OP is not equating contextual with insidious. They're pointing out, correctly, that contextual ads can be insidious. And if they're profitable, they probably will be. A lot of the companies offering LLM services are in a race gain market share and build expertise. Right now, they can burn through millions of dollars of VC money, with the expectation that they'll turn a profit at some point in the future. If that…

> If that profit comes from advertising, and critically, if users don't expect advertising in their free LLMs, because they didn't see ads in generated output in the past, that will be very insidious. Are the free LLM providers offering their service with a contractual obligation to the users that they will not add advertising to the outputs? If not, how is it insidious? What definition of insidious are you using per…

Weirdly, no part of that merriam-webster link includes the word "contract". I'm not sure you know how words work.

Re: LLMs are cheap

#280

Post by jackdeansmith »

cush wrote: post #241
xxbondsxx wrote: post #5

You can't compare an API that is profitable (search) to an API that is likely a loss-leader to grab market share (hosted LLM cloud models). Sure there might not be any analysis that proves that they subsidized, but you also don't have any evidence that they are profitable. All the data points we have today show that companies are spending an insane amount of capex on gaining AI dominance without the revenue to achiev…

> You can't compare an API that is profitable (search) to an API that is likely a loss-leader to grab market share (hosted LLM cloud models). Regardless of maturity lifecycle, by definition loss-leaders are cheap. If I go to the grocery store and milk is $1, I don't think I'm being swindled. I know it's a loss-leader and I buy it because it's cheap. We are currently in the early-Netflix-massive-library-for-five-dolla…

Want to bet? I'll give you 5:1 odds that tokens from a model with some specific benchmark performance (we can sort out the specific benchmarks or basket of benchmarks if you want to bet) will be cheaper two years from now.
Post reply on HN