Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Are the costs of AI agents also rising exponentially? (2025)

tobyord.com

Post reply on HN
Print view View original

11–20 of 155 posts

Re: Are the costs of AI agents also rising exponentially? (2025)

#11

Post by wavemode »

greenmilk wrote: post #4

Are any inference providers currently making profit (on inference, I know google makes money)?

Selling inference is not fundamentally different from selling compute - you amortize the lifetime cost of owning and operating the GPUs and then turn that into a per-token price. The risk of loss would be if there is low demand (and thus your facilities run underutilized), but I doubt inference providers are suffering from this.

Where the long-term payoff still seems speculative, is for companies doing training rather than just inference.

Re: Are the costs of AI agents also rising exponentially? (2025)

#12

Post by Gigachad »

wavemode wrote: post #11
greenmilk wrote: post #4

Are any inference providers currently making profit (on inference, I know google makes money)?

Selling inference is not fundamentally different from selling compute - you amortize the lifetime cost of owning and operating the GPUs and then turn that into a per-token price. The risk of loss would be if there is low demand (and thus your facilities run underutilized), but I doubt inference providers are suffering from this. Where the long-term payoff still seems speculative, is for companies doing training rathe…

There’s a lot of debate over what the useful lifespan of the hardware is though. A number that seems very vibes based determines if these datacenters are a good investment or disastrous.

Re: Are the costs of AI agents also rising exponentially? (2025)

#13

Post by colechristensen »

agentifysh wrote: post #10

Until there is some drastic new hardware, we are going to see a similar situation to proof of work, where a small group hordes the hardware and can collude on prices. Difference is that the current prices have a lot of subsidies from OPM Once the narrative changes to something more realistic, I can see prices increase across the board, I mean forget $200/month for codex pro, expect $1000/month or something similar. S…

Doubtful, local models are the competitive future that will keep prices down.

128GB is all you need.

A few more generations of hardware and open models will find people pretty happy doing whatever they need to on their laptop locally with big SOTA models left for special purposes. There will be a pretty big bubble burst when there aren't enough customers for $1000/month per seat needed to sustain the enormous datacenter models.

Apple will win this battle and nvidia will be second when their goals shift to workstations instead of servers.

Re: Are the costs of AI agents also rising exponentially? (2025)

#15

Post by thelastgallon »

> On many task lengths (including those near their plateau) they cost 10 to 100 times as much per hour. For instance, Grok 4 is at $0.40 per hour at its sweet spot, but $13 per hour at the start of its final plateau. GPT-5 is about $13 per hour for tasks that take about 45 minutes, but $120 per hour for tasks that take 2 hours. And o3 actually costs $350 per hour (more than the human price) to achieve tasks at its full 1.5 hour task horizon. This is a lot of money to pay for an agent that fails at the task you’ve just paid for 50% of the time — especially in cases where failure is much worse than not having tried at all.

Re: Are the costs of AI agents also rising exponentially? (2025)

#16

Post by lookaround »

colechristensen wrote: post #13
agentifysh wrote: post #10

Until there is some drastic new hardware, we are going to see a similar situation to proof of work, where a small group hordes the hardware and can collude on prices. Difference is that the current prices have a lot of subsidies from OPM Once the narrative changes to something more realistic, I can see prices increase across the board, I mean forget $200/month for codex pro, expect $1000/month or something similar. S…

Doubtful, local models are the competitive future that will keep prices down. 128GB is all you need. A few more generations of hardware and open models will find people pretty happy doing whatever they need to on their laptop locally with big SOTA models left for special purposes. There will be a pretty big bubble burst when there aren't enough customers for $1000/month per seat needed to sustain the enormous datacen…

> 128GB is all you need.

My guy, look around.

They are coming for personal compute.

Where are you going to get these 128GBs? Aquaman? [0]

The ones who make RAM are inexplicably attaching their fate to the future being all LLMs only everywhere.

[0] https://www.youtube.com/watch?v=0-w-pdqwiBw

Re: Are the costs of AI agents also rising exponentially? (2025)

#18

Post by naveen99 »

lookaround wrote: post #16
colechristensen wrote: post #13

Earlier quoted context omitted.

Doubtful, local models are the competitive future that will keep prices down. 128GB is all you need. A few more generations of hardware and open models will find people pretty happy doing whatever they need to on their laptop locally with big SOTA models left for special purposes. There will be a pretty big bubble burst when there aren't enough customers for $1000/month per seat needed to sustain the enormous datacen…

> 128GB is all you need. My guy, look around. They are coming for personal compute. Where are you going to get these 128GBs? Aquaman? [0] The ones who make RAM are inexplicably attaching their fate to the future being all LLMs only everywhere. [0] https://www.youtube.com/watch?v=0-w-pdqwiBw

Cloud can’t make money off of you and pay more than you for the hardware at the same time.

Re: Are the costs of AI agents also rising exponentially? (2025)

#19

Post by foota »

lookaround wrote: post #16
colechristensen wrote: post #13

Earlier quoted context omitted.

Doubtful, local models are the competitive future that will keep prices down. 128GB is all you need. A few more generations of hardware and open models will find people pretty happy doing whatever they need to on their laptop locally with big SOTA models left for special purposes. There will be a pretty big bubble burst when there aren't enough customers for $1000/month per seat needed to sustain the enormous datacen…

> 128GB is all you need. My guy, look around. They are coming for personal compute. Where are you going to get these 128GBs? Aquaman? [0] The ones who make RAM are inexplicably attaching their fate to the future being all LLMs only everywhere. [0] https://www.youtube.com/watch?v=0-w-pdqwiBw

More like RAM producers are providing supplies to the highest bidder, no? If this doesn't peter out supply will normalize at a higher but less insane price eventually.

Re: Are the costs of AI agents also rising exponentially? (2025)

#20

Post by hypercube33 »

colechristensen wrote: post #13
agentifysh wrote: post #10

Until there is some drastic new hardware, we are going to see a similar situation to proof of work, where a small group hordes the hardware and can collude on prices. Difference is that the current prices have a lot of subsidies from OPM Once the narrative changes to something more realistic, I can see prices increase across the board, I mean forget $200/month for codex pro, expect $1000/month or something similar. S…

Doubtful, local models are the competitive future that will keep prices down. 128GB is all you need. A few more generations of hardware and open models will find people pretty happy doing whatever they need to on their laptop locally with big SOTA models left for special purposes. There will be a pretty big bubble burst when there aren't enough customers for $1000/month per seat needed to sustain the enormous datacen…

Weird how you're leaving stuff like Strix Halo out. Also weird you think 128gb is the future with all of the research done to reduce that to something around 12GB being a target with all of these papers out now. I assume we'll end up with less general purpose models and more specific small ones swapped out for whatever work you are asking to do.
Post reply on HN