Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLMs are cheap

snellman.net

Post reply on HN
Print view View original

51–60 of 319 posts

Re: LLMs are cheap

#51

Post by andrew_lettuce »

dist-epoch wrote: post #39
pmdr wrote: post #33

I really doubt that, in an industry where chips are so hard to come by, draw so much power and are so terribly expensive, big players could at any time flip a switch and become profitable. They burn through insane amounts of cash and are, for some reason, still called startups. Sure, they'll be around for a long time until they figure something out, but unless hardware prices and power consumption go down, they won't…

Stock price go up is another way a company is profitable. The amazon playbook for 10+ years.

Amazon made huge money as they captured more and more of the market and didn't return any of it. The company literally became worth more and more each year. Open AI continues to hemorrhage money.

Re: LLMs are cheap

#52

Post by [deleted] »

Etheryte wrote: post #7

> OpenAI reportedly made a loss of $5B in 2024. They also reportedly have 500M MAUs. To reach break-even, they'd just need to monetize those free users for an average of $10/year, or $1/month. A $1 ARPU for a service like this would be pitifully low. This is a tangent to the rest of the article, but this "just" is doing more heavy lifting than Atlas holding up the skies. Taking a user from $0 to $1 is immeasurably ha…

[deleted]

Re: LLMs are cheap

#53

Post by johnisgood »

fastball wrote: post #23
prmoustache wrote: post #15

LLMs aren't cheap if you consider the impact on the climate and the cost that comes from it.

I didn't realize Large Language Models have a direct impact on the climate.

How about indirect? At any rate, something is going on, because our summers are more and more hotter, and there are no snow during our winters. We are all noticing it but it gets shrugged off as "misremembering". I am not contributing it to running LLMs alone, however, but climate change seems real enough to me, I experience it. It is barely July and I am dying! We used to have more tolerable weather around this time of year, for a long time.

Re: LLMs are cheap

#54

Post by thijson »

deadbabe wrote: post #19

Given how addicted people are to using LLMs steep price hikes are almost certainly guaranteed at some point. When this happens, what we will see is once again the rich and privileged will benefit from the use of LLMs while the poor have to just rely on their own brains. Consider how some students will have to grow up struggling through school without any LLMs while rich kids breeze their way through everything with t…

Not sure if you're being sarcastic. The cost of compute is perpetually going lower, it is getting harder to scale though. I feel like LLM's will become ubiquitous. When I went to University in the 90's, only the wealthy could afford cell phones, pulling one out was a flex. Now they are everywhere. Even Nvidia's sky high margins will someday be eroded.

Re: LLMs are cheap

#55

Post by AndrewDucker »

chaz6 wrote: post #29
Etheryte wrote: post #7

> OpenAI reportedly made a loss of $5B in 2024. They also reportedly have 500M MAUs. To reach break-even, they'd just need to monetize those free users for an average of $10/year, or $1/month. A $1 ARPU for a service like this would be pitifully low. This is a tangent to the rest of the article, but this "just" is doing more heavy lifting than Atlas holding up the skies. Taking a user from $0 to $1 is immeasurably ha…

I thought that services like these were run at a loss because the data that users provide is often worth more than the price of a subscription.

Only if you can find a way of monetisng that data or selling it on.

So, basically, ads.

Re: LLMs are cheap

#56

Post by exceptione »

theOGognf wrote: post #38
exceptione wrote: post #26

Earlier quoted context omitted.

This is only the power usage?

Right, this is only power usage. Factoring in labor and all that would make it more expensive for sure. However, it’s not like it’s a complex system to maintain. We use a popular inference server and just run it with some modest rate limits . It’s been hands-off for close to a year at this point

Ok! What hardware do you run? I had thought that would be the most expensive part.

Re: LLMs are cheap

#57

Post by paxys »

fkyoureadthedoc wrote: post #34
paxys wrote: post #20

The entire comparison hinges on people only making simple factual searches ("what is the capital of USA") on both search engines and LLMs. I'm going to say that's far enough from the standard use case for both these sets of APIs to be entirely meaningless. - If I'm using a search engine, I want to search the web. Yes these engines are increasingly providing answers rather than just search results, but that's a UI/pro…

Anecdotally, I'm a paying user and do a lot of super basic queries. What is this bug, rewrite this drivel into an email to my HOA, turn me into a gnome, what is the worst state and why is it west Virginia. This would probably increase 10x if one of the providers sold a family plan and my kids got paid access. Most of my heavy lifting is work related and goes through my employer's pockets.

None of those are "basic queries", in the sense that you will not be able to solve them using the Google/Bing search API.

Re: LLMs are cheap

#58

Post by dale_glass »

paxys wrote: post #20

The entire comparison hinges on people only making simple factual searches ("what is the capital of USA") on both search engines and LLMs. I'm going to say that's far enough from the standard use case for both these sets of APIs to be entirely meaningless. - If I'm using a search engine, I want to search the web. Yes these engines are increasingly providing answers rather than just search results, but that's a UI/pro…

I think the issue is that the classical search engine model has increasingly become less useful.

There's less experts using search engines. Normal people treat search engines less like an index search and more like a person. Asking an old school search engine "What is the capital of USA" is actually not quite right, because the "what is" is probably quite superfluous, and you're counting on finding some sort of educative website with the answer. In fact phrasing it as "the capital of the USA is" is probably a better fit for a search engine, since that's the sort of sentence that would contain what you want to know.

Also with the plague of "SEO", there's a million sites trying to convince Google that their site is relevant even when it's not.

So LLMs are increasingly more and more relevant at informally phrased queries that don't actually contain relevant key words, and they're also much more useful in that they bypass a lot of pointless verbiage, spam, ads and requests to subscribe.

Re: LLMs are cheap

#59

Post by relaxing »

bboygravity wrote: post #35
Etheryte wrote: post #7

> OpenAI reportedly made a loss of $5B in 2024. They also reportedly have 500M MAUs. To reach break-even, they'd just need to monetize those free users for an average of $10/year, or $1/month. A $1 ARPU for a service like this would be pitifully low. This is a tangent to the rest of the article, but this "just" is doing more heavy lifting than Atlas holding up the skies. Taking a user from $0 to $1 is immeasurably ha…

The entire businessmodel may only work as long as inference takes up the physical space and cost of a small building. Last time personal computing took up an entire building, we put the same compute power into a (portable) "personal computer" a few decades later. Can't wait to send all my data and life to my own lil inference box, instead of big tech (and NSA etc).

“Last time” we weren’t up against physical limitations for solid state electronics like the size of an atom, wavelength of light, quantum effects, thermal management, etc.

Re: LLMs are cheap

#60

Post by postexitus »

They are cheap as long as subsidized by VC and (in some cases) government money. We will see the real cost when free tiers disappear or start to be supported by ads.
Post reply on HN