Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLMs are cheap

snellman.net

Post reply on HN
Print view View original

231–240 of 319 posts

Re: LLMs are cheap

#231

Post by tecleandor »

fastball wrote: post #144
tecleandor wrote: post #48

Earlier quoted context omitted.

Well, running them does. And, from what I get from the article, that's what they're trying to do: either running them or having someone do it for them as a service. How big is that impact? Well, that's a complicated issue.

Running LLMs does not have any intrinsic impact on the climate. If you want to talk about the impact of different power generation methods on climate change, fair enough, but I don't think this thread is the place for it. Unless of course the idea is to talk about climate change in every single thread centered on "things that consumes energy", which is approximately all of them.

Ah no, I was just strictly defining OP's comment.

Of course, I was setting up a plain definition without magnitude. The impact could be near nil, or could be huge, but an impact nonetheless.

I didn't want to deep dive on it, as I though it would sidetrack the comments. I think this is a subject that merits certain analysis, as in some cases the discourse around the energy use has been akin to the one used in the highest moments of the past crypto/NFT cycle: "Yeah, it's not clean but it could."

And to that:

- It could, but it isn't (not only because of the popular gas turbines, but with collateral damage like the water use or heating, or by other social issues around it)

- But also (and this would be more philosophical/social), even if it were clean... is it worth it?

Re: LLMs are cheap

#233

Post by raydev »

jsnell wrote: post #208
rco8786 wrote: post #170

Earlier quoted context omitted.

Inference is cheap. Training is cheaper. Then where's all the money going? OpenAI is reporting heavy losses, but you're saying the unit economics of inference are all good. What are they spending money on?

Their spending is not a problem. It's quite low for a top-tier hard tech company that's also running a consumer service with 500M active users. They are making a loss because 95% of their users are on free accounts, and for now they're choosing not to monetize those users in any way (e.g. ads).

sama tweeted that the $200 tier was priced too low to cover costs a few months ago.

Re: LLMs are cheap

#234

Post by pigeons »

handfuloflight wrote: post #204
Workaccount2 wrote: post #199

Earlier quoted context omitted.

Because then the AI isn't working for you anymore, it's working for the advertisers. Which isn't necessarily bad, but we can be pretty confident that the AI will not be upfront about this, and instead try to act like it's working for you.

If the advertising is contextually relevant, how is it working against you?

Just being contextually relevant doesn't mean its in your interests as opposed to in the interests of the advertised or that the levers are transparent.

Re: LLMs are cheap

#235

Post by dcre »

bee_rider wrote: post #229
qeternity wrote: post #220

Earlier quoted context omitted.

> the old one is completely valueless to them This is of course untrue for the same reason that people are still running Windows 2000.

> This is of course untrue for the same reason that people are still running Windows 2000. What is the reason?

They’ve built processes around it and don’t feel like/can’t afford to/ don’t know to how change them.

Re: LLMs are cheap

#236

Post by burnte »

JimDabell wrote: post #101
xxbondsxx wrote: post #5

You can't compare an API that is profitable (search) to an API that is likely a loss-leader to grab market share (hosted LLM cloud models). Sure there might not be any analysis that proves that they subsidized, but you also don't have any evidence that they are profitable. All the data points we have today show that companies are spending an insane amount of capex on gaining AI dominance without the revenue to achiev…

> you also don't have any evidence that they are profitable. Sure we do. Go to AWS or any other hosting provider and pay them for inference. You think AWS are going to subsidise your usage of somebody else’s models indefinitely? > All the data points we have today show that companies are spending an insane amount of capex on gaining AI dominance without the revenue to achieve profitability yet. Yes, capex not opex. T…

Capex and opex are just accounting labels that help categorize costs and can improve planning ability. But at the end of the day a billion dollars is a billion dollars.

Re: LLMs are cheap

#237

Post by bee_rider »

dcre wrote: post #235
bee_rider wrote: post #229

Earlier quoted context omitted.

> This is of course untrue for the same reason that people are still running Windows 2000. What is the reason?

They’ve built processes around it and don’t feel like/can’t afford to/ don’t know to how change them.

I guess we’ll see how that shakes out.

Because models are getting much better every couple months, I wonder if getting too attached to a process built around one in particular is a bad idea.

Re: LLMs are cheap

#238

Post by Gerardo1 »

JimDabell wrote: post #101
xxbondsxx wrote: post #5

You can't compare an API that is profitable (search) to an API that is likely a loss-leader to grab market share (hosted LLM cloud models). Sure there might not be any analysis that proves that they subsidized, but you also don't have any evidence that they are profitable. All the data points we have today show that companies are spending an insane amount of capex on gaining AI dominance without the revenue to achiev…

> you also don't have any evidence that they are profitable. Sure we do. Go to AWS or any other hosting provider and pay them for inference. You think AWS are going to subsidise your usage of somebody else’s models indefinitely? > All the data points we have today show that companies are spending an insane amount of capex on gaining AI dominance without the revenue to achieve profitability yet. Yes, capex not opex. T…

> You think AWS are going to subsidies your usage of somebody else’s models

Yes

>indefinitely?

No, and that's the point.

Re: LLMs are cheap

#239

Post by codyb »

burnte wrote: post #236
JimDabell wrote: post #101

Earlier quoted context omitted.

> you also don't have any evidence that they are profitable. Sure we do. Go to AWS or any other hosting provider and pay them for inference. You think AWS are going to subsidise your usage of somebody else’s models indefinitely? > All the data points we have today show that companies are spending an insane amount of capex on gaining AI dominance without the revenue to achieve profitability yet. Yes, capex not opex. T…

Capex and opex are just accounting labels that help categorize costs and can improve planning ability. But at the end of the day a billion dollars is a billion dollars.

I believe there's a fair amount of tax implications involved with that bucketing though. Capex is taxed at a lower rate than opex is my understanding but I may be wrong on the specifics of it all.

Re: LLMs are cheap

#240

Post by handfuloflight »

pigeons wrote: post #234
handfuloflight wrote: post #204

Earlier quoted context omitted.

If the advertising is contextually relevant, how is it working against you?

Just being contextually relevant doesn't mean its in your interests as opposed to in the interests of the advertised or that the levers are transparent.

Are you assuming all commercial relationships are adversarial? Why can't advertisers and those advertised to have aligned interests? What transparent levers do non-advertised results have, how do you know search rankings don't have hidden commercial incentives? Why trust undisclosed bias over disclosed relationships? Isn't transparency about incentives better than pretending they don't exist?
Post reply on HN