Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Tokenomics: Quantifying Where Tokens Are Used in Agentic Software Engineering

arxiv.org

Post reply on HN
Print view View original

81–90 of 98 posts

Re: Tokenomics: Quantifying Where Tokens Are Used in Agentic Software Engineering

#81

Post by TSiege »

lucaspiller wrote: post #25
sedatk wrote: post #21

One month I could use Github Copilot fully with no disruptions. The next month, after pricing changes, I’ve run out of tokens in two days. Such drastic changes tell me that pricing of tokens is arbitrary, and AI business is running out of money fast.

I think it's more a consequence of pushing for the biggest valuation/IPO. Rumoured profits on inference are north of 70%. Taking SpaceX as an example, they have increased prices across all their consumer products over the past six months. But they definitely aren't short on money with Alphabet and Anthropic combined paying them over $2 billion per month. Microsoft/GitHub lost out here as they were just repacking othe…

How is spacex not short on money when no one will pay them to use their models and they lose money every quarter? Sure they’re now transitioning to a data center provider away from actually being an AI company because they’re losing less money that way but it doesn’t sound like a strategic success

Re: Tokenomics: Quantifying Where Tokens Are Used in Agentic Software Engineering

#82

Post by whattheheckheck »

monkeydust wrote: post #50

I have a MA system setup for personal use. You give it a problem, you then refine that problem where a fast, cheaper model asks you questions which you answer to get a better input prompt. You then choose a MA strategy for example take problem break up to sections then final judge concludes or you do multi turn where agents debate then judge summarises debate. The best approach is what I call 'all angles' where all t…

The cheap models may ask subpar questions leading to subpar solutions

Re: Tokenomics: Quantifying Where Tokens Are Used in Agentic Software Engineering

#83

Post by ethanwillis »

monkeydust wrote: post #50

I have a MA system setup for personal use. You give it a problem, you then refine that problem where a fast, cheaper model asks you questions which you answer to get a better input prompt. You then choose a MA strategy for example take problem break up to sections then final judge concludes or you do multi turn where agents debate then judge summarises debate. The best approach is what I call 'all angles' where all t…

I have also developed a similar system not focused on the exploratory refinement of prompt(s). But more focused on feedback loops cybernetic style, so focused on the maintaining of stability of the prompt outputs by a growing library of deterministic checks and autofixes. Anything that is a "problem" which isn't covered by that library is surfaced to the human driving the process.

Re: Tokenomics: Quantifying Where Tokens Are Used in Agentic Software Engineering

#84

Post by whattheheckheck »

NitpickLawyer wrote: post #76
shimman wrote: post #74

Earlier quoted context omitted.

No, you can't really make educated guesses unless people start opening their books. Especially in an industry where the vast majority of firms make up valuations out of thin air and not based on any reproducible insights.

Opening their books would let you know things like profitability. I'm talking about cost per token, model development and human costs being irrelevant.

Yeah take the gpu rental cost, what it can run, how many tokens per second come out and see the true rate per token. Plus the margin on harness special sauce

Re: Tokenomics: Quantifying Where Tokens Are Used in Agentic Software Engineering

#85

Post by flowbarai »

monkeydust wrote: post #55
uxhacker wrote: post #52

Earlier quoted context omitted.

So what harness are you using? And what LLM’s

Homebrew harness and all frontier ones plus deepseek. All via Openrouter at the moment. Works well enough but can get expensive so use for real high value challenges. Interestingly the refine feature has been most useful to me and people I have shown, essentially people are lazy when expressing the initial problem (me included!), refine asks relevant questions to initial problem then refines the initial statement, us…

[flagged]

Re: Tokenomics: Quantifying Where Tokens Are Used in Agentic Software Engineering

#88

Post by becomevocal »

jwnin wrote: post #65
zozbot234 wrote: post #35

Earlier quoted context omitted.

Code review could also be run as an unattended/batched task though, possibly with at least some use of on-prem inference (which excels at this). That would be a major saving compared to the usual cloud inference scenario.

with which models, though?

Yeah wasn’t there a report recently on how local models after energy costs didn’t weren’t actually more efficient to complete the same task?

Re: Tokenomics: Quantifying Where Tokens Are Used in Agentic Software Engineering

#89

Post by SOLAR_FIELDS »

monkeydust wrote: post #55
uxhacker wrote: post #52

Earlier quoted context omitted.

So what harness are you using? And what LLM’s

Homebrew harness and all frontier ones plus deepseek. All via Openrouter at the moment. Works well enough but can get expensive so use for real high value challenges. Interestingly the refine feature has been most useful to me and people I have shown, essentially people are lazy when expressing the initial problem (me included!), refine asks relevant questions to initial problem then refines the initial statement, us…

There is a 5 line skill I’ve been using for refinement called grill-me that works quite well
Post reply on HN