Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The current AI pricing was always going to go away

arnon.dk

Post reply on HN
Print view View original

81–90 of 100 posts

Re: The current AI pricing was always going to go away

#81

Post by kittikitti »

Thank you for sharing this article. I think the graphs in it were useful in understanding the different pricing structures. One thing that I would have included is pricing based on AI that I own, through capital expenditure (CapEx).

However, it's much harder to compare. For one, the cost per token is difficult to measure until a sufficient amount of time has passed so that an extrapolation is more accurate. Also, there are performance considerations where a local solution might be more or less accurate than an equivalent online AI. In addition, the reduced compliance risk is hard to quantify or it makes online AI practically useless.

I don't understand how people got buy-in for a business model that assumed token costs would go down indefinitely. All tech startups follow a blitz-scaling pattern where they practically give away their services for free, trap customers in a moat, and then extort as much money as they can.

Re: The current AI pricing was always going to go away

#82

Post by otabdeveloper4 »

JimDabell wrote: post #57
tekacs wrote: post #50

> Anthropic’s CFO testified under oath this March that the company spent $10 billion on compute and made $5 billion in revenue (Ed Zitron has the math). The labs are underwater on inference. They’re raising prices to keep the lights on. 'The labs are underwater on inference' is an absurd thing to say whilst not separating the cost of _compute_ out into training and inference.

According to Dario Amodei, Anthropic are even profitable when including inference as long as you look at it on a per-model basis; it’s just that every model is more expensive to train than the last one. For instance, if you have already spent $n to train a model and are currently earning $2n selling inference with it; but are concurrently spending $3n training the next model in anticipation of earning $6n with it, th…

> According to Dario Amodei

That's a big ask. No thanks.

Re: The current AI pricing was always going to go away

#83

Post by nightski »

gruez wrote: post #45
nightski wrote: post #20

Earlier quoted context omitted.

The labs have a perverse incentive to make things as expensive compute wise as possible. The only thing keeping this somewhat in check is competition, but it's intentionally being gatekept by locking up the supply of computing infrastructure. With 3 players it's pretty easy to collude even if indirectly. They can't burn trillions forever. Nvidia's 75% profit margins are not sustainable forever. Things will normalize,…

>The labs have a perverse incentive to make things as expensive compute wise as possible. The only thing keeping this somewhat in check is competition, but it's intentionally being gatekept by locking up the supply of computing infrastructure. With 3 players it's pretty easy to collude even if indirectly. By all accounts the AI capex boom is justified up by actual usage, rather than some nefarious plan for "locking u…

Right but that could be more evenly distributed. There is a circular trade right now giving these few players near infinite resources that is blocking that from happening.

Re: The current AI pricing was always going to go away

#85

Post by koliber »

pikminguy wrote: post #66
koliber wrote: post #61

EDIT: [ IGNORE THIS COMMENT -- IT IS WRONG - I had a "bad math moment" myself ] The math seems off. How is 7.8 million vs 4 million 95% more expensive. Article makes good points but I doubt the numbers as they don’t add up. Still agree with the conclusion though.

7.8 is higher than 4 by 3.8 3.8 is 95% of 4 The price went up by 3.8 or in other words the price went up by 95%. I'm not sure what math you're not mathing.

Absolutely right. I had a "math moment" myself. I was thinking margin instead of markup.

Re: The current AI pricing was always going to go away

#86

Post by tlb »

NitpickLawyer wrote: post #67
clhodapp wrote: post #31

Earlier quoted context omitted.

I know it comes off as pedantic to point this out but: Those are open weight models not open source models. Closed weight models are the equivalent of SaaS. Open weight models are the equivalent of binary driver blobs or Windows software. We don't really have actual open source LLMs, which would need to publicly release their training data and technique so you could train a similar model yourself, or use their work a…

I know this is highly contested, but I'll try explaining it anyway, because I keep seeing this and it's ... wrong. Your comment is wrong both theoretically and practically. First, the theory. The idea that model weights are "binary driver blobs" is technically wrong. I don't know why this is so common on a technical site, but anyway. An LLM model consists of 3 main parts: The architecture, the inference code, and som…

There are still things you can't do with an open-weight model without the training data, like modifying the architecture and training from scratch. That's different from true open-source code, where you can do anything the authors could do.

Re: The current AI pricing was always going to go away

#90

Post by charcircuit »

NitpickLawyer wrote: post #67
clhodapp wrote: post #31

Earlier quoted context omitted.

I know it comes off as pedantic to point this out but: Those are open weight models not open source models. Closed weight models are the equivalent of SaaS. Open weight models are the equivalent of binary driver blobs or Windows software. We don't really have actual open source LLMs, which would need to publicly release their training data and technique so you could train a similar model yourself, or use their work a…

I know this is highly contested, but I'll try explaining it anyway, because I keep seeing this and it's ... wrong. Your comment is wrong both theoretically and practically. First, the theory. The idea that model weights are "binary driver blobs" is technically wrong. I don't know why this is so common on a technical site, but anyway. An LLM model consists of 3 main parts: The architecture, the inference code, and som…

The inference code is not part of a LLM and there can be multiple different implementations of it. The model, code to train the model, and code to run the modal are different things.
Post reply on HN