Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Are the costs of AI agents also rising exponentially? (2025)

tobyord.com

Post reply on HN
Print view View original

141–150 of 155 posts

Re: Are the costs of AI agents also rising exponentially? (2025)

#141

Post by pindab0ter »

smusamashah wrote: post #82
radialstub wrote: post #77

Earlier quoted context omitted.

No, burning models into hardware won't make them faster or reduce the cost. It will cost way more for similar performance as what you would get with a gpu. I am not telling you why, you can go figure that out on your own.

But isn't this happening here https://taalas.com/ already. They have a demo of llama running at 17000 tokens per second https://chatjimmy.ai/

I just tried chatjimmy.ai for a bit and while it is absolutely blazingly fast, it's also not a very strong model. I suppose that with time, stronger models will be able to run on such hardware, too.

Re: Are the costs of AI agents also rising exponentially? (2025)

#142

Post by robkop »

gjsman-1000 wrote: post #88
smusamashah wrote: post #82

Earlier quoted context omitted.

But isn't this happening here https://taalas.com/ already. They have a demo of llama running at 17000 tokens per second https://chatjimmy.ai/

With some research, that chip appears like it would cost about $300-$400 to manufacture, die only. For an 8B parameter model. Opus is estimated at 500B-2T parameters. At that scale you’re past reticle limits and need HBM and multi-die packaging, which means you’ve essentially built an inference ASIC (like Groq or Etched) rather than something categorically cheaper than GPUs. The “burned into silicon” advantage mostly…

There’s a lot of tradeoffs to play with, those inference ASICs may not carry the gradient but they are still optimised for larger batches and to run any model. They need enough memory for the weights, wide batch inference, and ideally leftovers for kv cache efficiency.

For personal inference you’re given a lot more room to play in - much of it poorly explored today - enough to concern an argument of cost advantages evaporating

Re: Are the costs of AI agents also rising exponentially? (2025)

#143

Post by mixermachine »

zozbot234 wrote: post #111
mixermachine wrote: post #108

Earlier quoted context omitted.

The cutting edge, max size models will likely stay in the GPU space for a long time. But these models are not needed for most general requests. With a fine tuned 30B quantisized model you can serve a large portion of requests with around 32GB of RAM. Free users will likely only get these kinds of models. At some point we will get these models in hardware and the cost per token will be minimal.

> With a fine tuned 30B quantisized model you can serve a large portion of requests with around 32GB of RAM. Free users will likely only get these kinds of models. These are exactly the kinds of models that you can easily run locally by repurposing existing hardware. Depending on how much you're willing to wait for the answer, running local even gives you strictly better outcomes for simple Q&A queries. (Long-context…

For programmers maybe. I do this too. But think about all the regular users out there. Your dad and your mum, maybe even your grandparents. This is a huge marked too and for that we can use these special chips at scale.

Re: Are the costs of AI agents also rising exponentially? (2025)

#144

Post by mike_hearn »

zipy124 wrote: post #138
rcbdev wrote: post #125

Earlier quoted context omitted.

Software developers average about 80 - 120€/h in Austria. How are UK rates so bad?

A quick google seems to suggest the median software pay in Austria is about €60k. That translates to an hourly wage of roughly 31€/h . Your range of 80-120 gives a yearly wage of like 155k-234k which is clearly far too high for an average.

The discrepancy is probably consulting vs salaried.

Re: Are the costs of AI agents also rising exponentially? (2025)

#145

Post by Grimblewald »

onchainintel wrote: post #76
ting0 wrote: post #60

No, but the AI labs would love to frame it this way so they can continue to nerf models and increase prices while they use the cheap, highly performant, highly powerful models internally to replace all of your businesses.

Sure is looking that way. What can't Claude do at this point?

List is massive. Anything novel? It'll fuck up without extreme handholding. Anything for which the components arent solved public published problems? It'll fuck it up.

Basically, claude can solve issues for you where it requires the implementation of existing code or a combination of existing patterns, but novel it cannot do.

Re: Are the costs of AI agents also rising exponentially? (2025)

#146

Post by ai-x »

dminik wrote: post #122
ai-x wrote: post #115

Earlier quoted context omitted.

Math / Economics 101 thought experiment. You have (limited) 100 Coke cans to sell (that you bought for say $1) There are two large lines being formed for that. One line is offering an average $3 per bottle and another line is offering an average $2 per bottle. Tell me which line they would throttle/starve even though they make a profit out of it. Also, when the lines were formed you had no idea of the average price,…

If I owned two lines both selling the same thing (preumably here Coke is a stand-in for compute), I would throttle the $2 dollar line. People without a choice might move to the $3 dollar line. Unfortunately, back in the real world, Anthropic is dealing with two issues: 1. They're throttling all lines. Their latest model uses more tokens overall. Tokens are being rationed and context is being lowered. 2. There's anoth…

for all you know, you think you are standing in the $3 line, but it really is $2 line and $3 line is BigCos, Govt and others who have guaranteed demand for several years.

Re: Are the costs of AI agents also rising exponentially? (2025)

#147

Post by rcbdev »

zipy124 wrote: post #138
rcbdev wrote: post #125

Earlier quoted context omitted.

Software developers average about 80 - 120€/h in Austria. How are UK rates so bad?

A quick google seems to suggest the median software pay in Austria is about €60k. That translates to an hourly wage of roughly 31€/h . Your range of 80-120 gives a yearly wage of like 155k-234k which is clearly far too high for an average.

If you start talking about salaried work, you have to account for Austria's 5 weeks of mandatory PTO, infinite paid sick days (with a pay reduction after 6 weeks, clock resets each year), additional paid care leave (Pflegefreistellung) for sick family, paid maternity / paternity leave, etc. in contrast with whatever the British system offers. For developers in Austria who choose salaried work, the math tends to add up with at least 2 months off each year, union protection etc.

You'd have to be an expert on labor law in every country you seek to honestly compare. The only clean way to simply compare earnings is freelance rates.

Re: Are the costs of AI agents also rising exponentially? (2025)

#148

Post by greenmilk »

henry2023 wrote: post #17
greenmilk wrote: post #4

Are any inference providers currently making profit (on inference, I know google makes money)?

Third parties selling open-weight inference on OpenRouter are surely selling on a profit. Zero reason to subsidize it.

They could be VC-backed and selling at a loss to grow market share

Re: Are the costs of AI agents also rising exponentially? (2025)

#149

Post by GorbachevyChase »

avidiax wrote: post #97
GorbachevyChase wrote: post #93

Earlier quoted context omitted.

I have a lot of AI written software, and it doesn’t cost me anywhere close to what I’ve been quoted for other software projects in the past. I’ve had a guy spend over six months, full-time, on a CRUD application for permits. He didn’t even finish. I made a working prototype in Django, which was tossed to re-implement in PHP for some reason.

My understanding is that this is normalized to the "best human" for the tasks. An AI only doing a task correctly 50% of the time may in-fact be better than your N% chance of hiring a highly capable human for that task, and especially for contracting a human to a 1-2 hour task. But your successful use of AI is still predicated on a human who can judge output and break the work into smaller tasks that fit the skill cei…

False. That guy spent 6 months goofing off for something I can have in minutes. Nothing is a 1-2 hour task for a human working outside of a hobby. Procurement will involve half a dozen people at least and hours of meetings and scope development. And what quality assurance is provided in an hour? Your word? And this is better than my own vibes?

I just want to put this plainly. Programmers are often completely delusional about their own productivity and nobody can reliably define work quality in the field. The whole field is a mess, and I’m glad product owners and users are finally being empowered.

Re: Are the costs of AI agents also rising exponentially? (2025)

#150

Post by zipy124 »

rcbdev wrote: post #147
zipy124 wrote: post #138

Earlier quoted context omitted.

A quick google seems to suggest the median software pay in Austria is about €60k. That translates to an hourly wage of roughly 31€/h . Your range of 80-120 gives a yearly wage of like 155k-234k which is clearly far too high for an average.

If you start talking about salaried work, you have to account for Austria's 5 weeks of mandatory PTO, infinite paid sick days (with a pay reduction after 6 weeks, clock resets each year), additional paid care leave (Pflegefreistellung) for sick family, paid maternity / paternity leave, etc. in contrast with whatever the British system offers. For developers in Austria who choose salaried work, the math tends to add u…

Even consulting/free-lance compared to employed, the difference is not 3x...
Post reply on HN