Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Tokenomics: Quantifying Where Tokens Are Used in Agentic Software Engineering

arxiv.org

Post reply on HN
Print view View original

31–40 of 98 posts

Re: Tokenomics: Quantifying Where Tokens Are Used in Agentic Software Engineering

#31

Post by emsign »

gmerc wrote: post #28

It’s just like Airline reward miles and offers no benefit to companies over just renting bare metal GPU time

I hope this horrible time will soon be over when cheaper NPUs come available from more hardware companies, and also when model size get optimized down further.

I wonder what hyperscaled compute farms and models will be good for at that running cost when most AI needs can be fulfilled by on-prem and on-device hardware and models. Probably only customer left are big governments. So in the end the tax payer has to pay for those billions of investments by the AI cartel.

Re: Tokenomics: Quantifying Where Tokens Are Used in Agentic Software Engineering

#33

Post by zozbot234 »

emsign wrote: post #31
gmerc wrote: post #28

It’s just like Airline reward miles and offers no benefit to companies over just renting bare metal GPU time

I hope this horrible time will soon be over when cheaper NPUs come available from more hardware companies, and also when model size get optimized down further. I wonder what hyperscaled compute farms and models will be good for at that running cost when most AI needs can be fulfilled by on-prem and on-device hardware and models. Probably only customer left are big governments. So in the end the tax payer has to pay f…

The typical NPU is only marginally helpful for on-prem inference. A GPU can read quantized data from main memory and dequantize/pad it locally (making effective use of memory throughput); a NPU often needs to read padded data directly from memory, which is wasteful. So it only helps a little bit wrt. prefill.

Also, smaller models can obviously be used but a smaller model will be a lot weaker in real-world knowledge and this tends to limit their smarts in a way that can't be compensated by more thinking.

Re: Tokenomics: Quantifying Where Tokens Are Used in Agentic Software Engineering

#34

Post by prymitive »

senectus1 wrote: post #15

amusing side note: Was in a meeting reviewing a potential new product, it was going well until they showed us that they had added AI to it (of course they have). It was pretty obviously just shoehorned in, and one part of that obviousness was that they had a column that showed how many tokens it took to make each query. I asked who is paying for the tokens, they said its included in the license. I said, so is there a…

AIshittification

Re: Tokenomics: Quantifying Where Tokens Are Used in Agentic Software Engineering

#35

Post by zozbot234 »

becomevocal wrote: post #32

First thought was "only 30 tasks" however the findings map to what I've seen personally: code review consumes majority of tokens

Code review could also be run as an unattended/batched task though, possibly with at least some use of on-prem inference (which excels at this). That would be a major saving compared to the usual cloud inference scenario.

Re: Tokenomics: Quantifying Where Tokens Are Used in Agentic Software Engineering

#36

Post by dkersten »

satvikpendem wrote: post #10

Tokenomics is already a word used to describe cryptocurrency economics, not sure why they'd try to redefine it for AI even if a different sort of token is used.

Crypto was already a term before cryptocurrencies made it about them. Web 3.0 was already a thing before crypto bros made web 3 about cryptocurrencies.

So what? Terms are reused in different contexts all the time. And most people have moved on from cryptocurrencies anyway, so there’s little chance it’ll confuse anyone.

Re: Tokenomics: Quantifying Where Tokens Are Used in Agentic Software Engineering

#39

Post by oersted »

ares623 wrote: post #18
Retric wrote: post #7

Earlier quoted context omitted.

That assumes Tokens will remain a meaningful expense. I’m not sure developers will find uses for ever more tokens nearly as quickly as the prices fall.

How are we so confident that prices will fall? Isn't the exact opposite happening, right now, during arguably the most critical part of this whole saga (pre-IPO to make things appear as beautiful and as not-obviously-illegal as possible)? And the only reason they were "falling" previously was for hyper growth.

Prices have fallen dramatically over the last few years. It’s just that our standards have increased because we are using AI in ways that were not possible with worse models. But for the same level of “intelligence” as we had a couple years ago, the prices are so much lower.

Re: Tokenomics: Quantifying Where Tokens Are Used in Agentic Software Engineering

#40

Post by jurgenburgen »

lucaspiller wrote: post #25
sedatk wrote: post #21

One month I could use Github Copilot fully with no disruptions. The next month, after pricing changes, I’ve run out of tokens in two days. Such drastic changes tell me that pricing of tokens is arbitrary, and AI business is running out of money fast.

I think it's more a consequence of pushing for the biggest valuation/IPO. Rumoured profits on inference are north of 70%. Taking SpaceX as an example, they have increased prices across all their consumer products over the past six months. But they definitely aren't short on money with Alphabet and Anthropic combined paying them over $2 billion per month. Microsoft/GitHub lost out here as they were just repacking othe…

> Rumoured profits on inference are north of 70%.

Rumors are worth squat when they’re most likely put in motion by the people with a vested interest in this industry.

Let’s talk about profits when there’s real data from the IPO documentation.

Post reply on HN