Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Are the costs of AI agents also rising exponentially? (2025)

tobyord.com

Post reply on HN
Print view View original

31–40 of 155 posts

Re: Are the costs of AI agents also rising exponentially? (2025)

#31

Post by boxedemp »

thelastgallon wrote: post #15

> On many task lengths (including those near their plateau) they cost 10 to 100 times as much per hour. For instance, Grok 4 is at $0.40 per hour at its sweet spot, but $13 per hour at the start of its final plateau. GPT-5 is about $13 per hour for tasks that take about 45 minutes, but $120 per hour for tasks that take 2 hours. And o3 actually costs $350 per hour (more than the human price) to achieve tasks at its fu…

If you gave me an agent that succeeded 50% of tasks I gave it, I could take over the world in a week. Faster if I wasn't so lazy.

I think you're overestimating, or oversimplifying. Maybe both.

Re: Are the costs of AI agents also rising exponentially? (2025)

#32

Post by jiggawatts »

agentifysh wrote: post #10

Until there is some drastic new hardware, we are going to see a similar situation to proof of work, where a small group hordes the hardware and can collude on prices. Difference is that the current prices have a lot of subsidies from OPM Once the narrative changes to something more realistic, I can see prices increase across the board, I mean forget $200/month for codex pro, expect $1000/month or something similar. S…

> Until there is some drastic new hardware

For inference, there is already a 10x improvement possible over a setup based on NVIDIA server GPUs, but volume production, etc... will take a while to catch up.

During inference the model weights are static, so they can be stored in High Bandwidth Flash (HBF) instead of High Bandwidth Memory (HBM). Flash chips are being made with over 300 layers and they use a fraction of the power compared to DRAM.

NVIDIA GPUs are general purpose. Sure, they have "tensor cores", but that's a fraction of the die area. Google's TPUs are much more efficient for inference because they're mostly tensor cores by area, which is why Gemini's pricing is undercutting everybody else despite being a frontier model.

New silicon process nodes are coming from TSMC, Intel, and Samsung that should roughly double the transistor density.

There's also algorithmic improvements like the recently announced Google TurboQuant.

Not to mention that pure inference doesn't need the crazy fast networking that training does, or the storage, or pretty much anything other than the tensor units and a relatively small host server that can send a bit of text back and forth.

Re: Are the costs of AI agents also rising exponentially? (2025)

#33

Post by Tepix »

MrBuddyCasino wrote: post #30
hypercube33 wrote: post #20

Earlier quoted context omitted.

Weird how you're leaving stuff like Strix Halo out. Also weird you think 128gb is the future with all of the research done to reduce that to something around 12GB being a target with all of these papers out now. I assume we'll end up with less general purpose models and more specific small ones swapped out for whatever work you are asking to do.

Strix Halo hasn‘t got nearly enough bandwidth, its just 256bit.

It‘s sufficient for some MoE models.

Re: Are the costs of AI agents also rising exponentially? (2025)

#34

Post by zozbot234 »

jiggawatts wrote: post #32
agentifysh wrote: post #10

Until there is some drastic new hardware, we are going to see a similar situation to proof of work, where a small group hordes the hardware and can collude on prices. Difference is that the current prices have a lot of subsidies from OPM Once the narrative changes to something more realistic, I can see prices increase across the board, I mean forget $200/month for codex pro, expect $1000/month or something similar. S…

> Until there is some drastic new hardware For inference, there is already a 10x improvement possible over a setup based on NVIDIA server GPUs, but volume production, etc... will take a while to catch up. During inference the model weights are static, so they can be stored in High Bandwidth Flash (HBF) instead of High Bandwidth Memory (HBM). Flash chips are being made with over 300 layers and they use a fraction of t…

> Flash chips are being made with over 300 layers and they use a fraction of the power compared to DRAM.

Isn't reading from flash significantly more power intensive than reading DRAM? Anyway, the overhead of keeping weights in memory becomes negligible at scale because you're running large batches and sharding a single model over large amounts of GPU's. (And that needs the crazy fast networking to make it work, you get too much latency otherwise.)

Re: Are the costs of AI agents also rising exponentially? (2025)

#35

Post by nopinsight »

thelastgallon wrote: post #15

> On many task lengths (including those near their plateau) they cost 10 to 100 times as much per hour. For instance, Grok 4 is at $0.40 per hour at its sweet spot, but $13 per hour at the start of its final plateau. GPT-5 is about $13 per hour for tasks that take about 45 minutes, but $120 per hour for tasks that take 2 hours. And o3 actually costs $350 per hour (more than the human price) to achieve tasks at its fu…

Ord's frontier-cost argument is right as far as it goes, but the piece doesn't engage with the counter-trend: inference cost for a fixed capability level has been falling faster than Moore's law. Pushing the frontier will likely keep getting more expensive and concentrated among a few players, while the intelligence needed for more mundane tasks keeps getting cheaper.

That raises a question: if practical-tier inference commoditizes, how does any company justify the ever-larger capex to push the frontier?

OpenAI's pitch is that their business model should "scale with the value intelligence delivers." Concretely, that means moving beyond API fees into licensing and outcome-based pricing in high-value R&D sectors like drug discovery and materials science, where a single breakthrough dwarfs compute cost. That's one possible answer, though it's unclear whether the mechanism will work in practice.

Re: Are the costs of AI agents also rising exponentially? (2025)

#36

Post by raincole »

boxedemp wrote: post #31
thelastgallon wrote: post #15

> On many task lengths (including those near their plateau) they cost 10 to 100 times as much per hour. For instance, Grok 4 is at $0.40 per hour at its sweet spot, but $13 per hour at the start of its final plateau. GPT-5 is about $13 per hour for tasks that take about 45 minutes, but $120 per hour for tasks that take 2 hours. And o3 actually costs $350 per hour (more than the human price) to achieve tasks at its fu…

If you gave me an agent that succeeded 50% of tasks I gave it, I could take over the world in a week. Faster if I wasn't so lazy. I think you're overestimating, or oversimplifying. Maybe both.

No one is claiming an agent can do 50% of arbitrary tasks. It's just 50% of METR's benchmark set.

> I think you're overestimating, or oversimplifying

Yeah if you only read comments on HN but not the actual linked article you will get oversimplified conclusion. Like, duh?

Re: Are the costs of AI agents also rising exponentially? (2025)

#37

Post by boxedemp »

raincole wrote: post #36
boxedemp wrote: post #31

Earlier quoted context omitted.

If you gave me an agent that succeeded 50% of tasks I gave it, I could take over the world in a week. Faster if I wasn't so lazy. I think you're overestimating, or oversimplifying. Maybe both.

No one is claiming an agent can do 50% of arbitrary tasks. It's just 50% of METR's benchmark set. > I think you're overestimating, or oversimplifying Yeah if you only read comments on HN but not the actual linked article you will get oversimplified conclusion. Like, duh?

Sorry for stating something so obvious. I'll comment less from now on.

Re: Are the costs of AI agents also rising exponentially? (2025)

#38

Post by ai-x »

quicklywilliam wrote: post #9

Interesting read. I don't know if I quite buy the evidence, but it's definitely enough to warrant further investigation. It also matches up with my personal experience, which is that tools like Claude Code are burning through more and more tokens as we push them to do bigger and bigger work. But we all know the frontier model companies are burning through money in an unsustainable race to get you and your company hoo…

Anthropic has 50% gross margins on their tokens.

Step 1) Bubble callers will be proven wrong in 2026 if not already (no excess capacity)

Step 2) Models are not profitable are proven wrong (When Anthropic files their S1)

Step 3) FOMO and actual bubble (say around 2028/29)

Re: Are the costs of AI agents also rising exponentially? (2025)

#39

Post by easygenes »

While I understand why they used the METR data, a cleaner look would be against the current cost-optimal frontier of open models (e.g. GLM-5.1 and MiniMax-M2.7). That paints a very different picture. Comparing just the frontier models at the time of the METR report invariably leads to looking at providers who are pushing the limits of cost at the time of the report.

GPT-5 was shown as being on the costly end, surpassed by o3 at over $100/hr. I can't directly compare to METR's metrics, but a good proxy is the cost of the Artificial Analysis suite. GLM-5.1 is less than half the cost to complete the suite of GPT-5 and is dramatically more capable than both GPT-5 and o3.

So while their analysis is interesting, it points towards the frontier continuing to test the limits of acceptable pricing (as Mythos is clearly reinforcing) and the lagging 6-12 months of distillation and refinement continuing to bring the cost of comparable capabilities to much more reasonable levels.

Re: Are the costs of AI agents also rising exponentially? (2025)

#40

Post by zozbot234 »

thelastgallon wrote: post #15

> On many task lengths (including those near their plateau) they cost 10 to 100 times as much per hour. For instance, Grok 4 is at $0.40 per hour at its sweet spot, but $13 per hour at the start of its final plateau. GPT-5 is about $13 per hour for tasks that take about 45 minutes, but $120 per hour for tasks that take 2 hours. And o3 actually costs $350 per hour (more than the human price) to achieve tasks at its fu…

This effect is likely even larger when you consider that the raw cost per inferred token grows linearly with context, rather than being constant. So longer tasks performed with higher-context models will cost quadratically more. The computational cost also grows super-linearly with model parameter size: a 20B-active model is more than four times the cost of a 5B-active model.
Post reply on HN