Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Are the costs of AI agents also rising exponentially? (2025)

tobyord.com

Post reply on HN
Print view View original

51–60 of 155 posts

Re: Are the costs of AI agents also rising exponentially? (2025)

#51

Post by zozbot234 »

jiggawatts wrote: post #50
zozbot234 wrote: post #34

Earlier quoted context omitted.

> Flash chips are being made with over 300 layers and they use a fraction of the power compared to DRAM. Isn't reading from flash significantly more power intensive than reading DRAM? Anyway, the overhead of keeping weights in memory becomes negligible at scale because you're running large batches and sharding a single model over large amounts of GPU's. (And that needs the crazy fast networking to make it work, you g…

For a given capacity of memory, Flash uses far less power than DRAM, especially when used mostly for reads. > becomes negligible at scale Nothing is negligible at scale! Both the cost and power draw of the HBMs is a limiting factor for the hyperscalers, to the point that Sam Altman (famously!) cornered the market and locked in something like 40% of global RAM production, driving up prices for everyone. > sharding a s…

> For a given capacity of memory, Flash uses far less power than DRAM, especially when used mostly for reads.

Flash has no idle power being non-volatile (whereas DRAM has refresh) but active power for reading a constantly-sized block is significantly larger for Flash. You can still use Flash profitably, but only for rather sparse and/or low-intensity reads. That probably fits things like MoE layers if the MoE is sparse enough.

Also, you can't really use flash memory (especially soldered-in HBF) for ephemeral data like the KV context for a single inference, it wears out way too quickly.

Re: Are the costs of AI agents also rising exponentially? (2025)

#52

Post by jurgenburgen »

boxedemp wrote: post #31
thelastgallon wrote: post #15

> On many task lengths (including those near their plateau) they cost 10 to 100 times as much per hour. For instance, Grok 4 is at $0.40 per hour at its sweet spot, but $13 per hour at the start of its final plateau. GPT-5 is about $13 per hour for tasks that take about 45 minutes, but $120 per hour for tasks that take 2 hours. And o3 actually costs $350 per hour (more than the human price) to achieve tasks at its fu…

If you gave me an agent that succeeded 50% of tasks I gave it, I could take over the world in a week. Faster if I wasn't so lazy. I think you're overestimating, or oversimplifying. Maybe both.

> If you gave me an agent that succeeded 50% of tasks I gave it, I could take over the world in a week. Faster if I wasn't so lazy.

Assuming you used o3, that would cost $58800 per week. That’s an expensive bet for only 50% odds in your favor.

Of course the agents are only that good on benchmarks, in reality your odds are worse. Maybe roulette instead?

Re: Are the costs of AI agents also rising exponentially? (2025)

#53

Post by 2848484995 »

ai-x wrote: post #38
quicklywilliam wrote: post #9

Interesting read. I don't know if I quite buy the evidence, but it's definitely enough to warrant further investigation. It also matches up with my personal experience, which is that tools like Claude Code are burning through more and more tokens as we push them to do bigger and bigger work. But we all know the frontier model companies are burning through money in an unsustainable race to get you and your company hoo…

Anthropic has 50% gross margins on their tokens. Step 1) Bubble callers will be proven wrong in 2026 if not already (no excess capacity) Step 2) Models are not profitable are proven wrong (When Anthropic files their S1) Step 3) FOMO and actual bubble (say around 2028/29)

Can we see them?

Re: Are the costs of AI agents also rising exponentially? (2025)

#56

Post by adrian_b »

zozbot234 wrote: post #51
jiggawatts wrote: post #50

Earlier quoted context omitted.

For a given capacity of memory, Flash uses far less power than DRAM, especially when used mostly for reads. > becomes negligible at scale Nothing is negligible at scale! Both the cost and power draw of the HBMs is a limiting factor for the hyperscalers, to the point that Sam Altman (famously!) cornered the market and locked in something like 40% of global RAM production, driving up prices for everyone. > sharding a s…

> For a given capacity of memory, Flash uses far less power than DRAM, especially when used mostly for reads. Flash has no idle power being non-volatile (whereas DRAM has refresh) but active power for reading a constantly-sized block is significantly larger for Flash. You can still use Flash profitably, but only for rather sparse and/or low-intensity reads. That probably fits things like MoE layers if the MoE is spar…

Modern flash memory, with multi-bit cells, indeed requires more power for reading than DRAM, for the same amount of data.

However, for old-style 1-bit per cell flash memory I do not see any reason for differences in power consumption for reading.

Different array designs and sense amplifier designs and CMOS fabrication processes can result in different power consumptions, but similar techniques can be applied to both kinds of memories for reducing the power consumption.

Of course, storing only 1 bit per cell instead of 3 or 4 reduces a lot the density and cost advantages of flash memory, but what remains may still be enough for what inference needs.

Re: Are the costs of AI agents also rising exponentially? (2025)

#57

Post by dannersy »

greenmilk wrote: post #4

Are any inference providers currently making profit (on inference, I know google makes money)?

If they were they would show evidence because they'd pull in more investment. I don't believe their claim that they make profits on inference, especially not with reports like this coming out.

Re: Are the costs of AI agents also rising exponentially? (2025)

#58

Post by TeMPOraL »

raincole wrote: post #36
boxedemp wrote: post #31

Earlier quoted context omitted.

If you gave me an agent that succeeded 50% of tasks I gave it, I could take over the world in a week. Faster if I wasn't so lazy. I think you're overestimating, or oversimplifying. Maybe both.

No one is claiming an agent can do 50% of arbitrary tasks. It's just 50% of METR's benchmark set. > I think you're overestimating, or oversimplifying Yeah if you only read comments on HN but not the actual linked article you will get oversimplified conclusion. Like, duh?

> Yeah if you only read comments on HN but not the actual linked article you will get oversimplified conclusion. Like, duh?

Curiously, for most submissions it's the opposite - comments are much more useful and nuanced than the source being discussed.

Post reply on HN