[flagged]
Tokenomics: Quantifying Where Tokens Are Used in Agentic Software Engineering
71–80 of 98 posts
Re: Tokenomics: Quantifying Where Tokens Are Used in Agentic Software Engineering
#72One month I could use Github Copilot fully with no disruptions. The next month, after pricing changes, I’ve run out of tokens in two days. Such drastic changes tell me that pricing of tokens is arbitrary, and AI business is running out of money fast.
I think it's more a consequence of pushing for the biggest valuation/IPO. Rumoured profits on inference are north of 70%. Taking SpaceX as an example, they have increased prices across all their consumer products over the past six months. But they definitely aren't short on money with Alphabet and Anthropic combined paying them over $2 billion per month. Microsoft/GitHub lost out here as they were just repacking othe…
SpaceX is increasing prices because they're trying really hard to get into the S&P 500.
Re: Tokenomics: Quantifying Where Tokens Are Used in Agentic Software Engineering
#73[dead]
Re: Tokenomics: Quantifying Where Tokens Are Used in Agentic Software Engineering
#74Earlier quoted context omitted.
> Rumoured profits on inference are north of 70%. Rumors are worth squat when they’re most likely put in motion by the people with a vested interest in this industry. Let’s talk about profits when there’s real data from the IPO documentation.
> Rumors are worth squat You can make some educated guesses and find out some limits on inferencing cost by looking at 3rd party providers on platforms like openrouter. You can get some median cost /tok for a given model size. Then make some educated guesses on SotA model sizes, and you can get an estimate on pure cost of serving a model. Error bars and all that, of course. But still a range, with some limits.
No, you can't really make educated guesses unless people start opening their books. Especially in an industry where the vast majority of firms make up valuations out of thin air and not based on any reproducible insights.
Re: Tokenomics: Quantifying Where Tokens Are Used in Agentic Software Engineering
#75One month I could use Github Copilot fully with no disruptions. The next month, after pricing changes, I’ve run out of tokens in two days. Such drastic changes tell me that pricing of tokens is arbitrary, and AI business is running out of money fast.
I think it's more a consequence of pushing for the biggest valuation/IPO. Rumoured profits on inference are north of 70%. Taking SpaceX as an example, they have increased prices across all their consumer products over the past six months. But they definitely aren't short on money with Alphabet and Anthropic combined paying them over $2 billion per month. Microsoft/GitHub lost out here as they were just repacking othe…
[deleted]
Re: Tokenomics: Quantifying Where Tokens Are Used in Agentic Software Engineering
#76Earlier quoted context omitted.
> Rumors are worth squat You can make some educated guesses and find out some limits on inferencing cost by looking at 3rd party providers on platforms like openrouter. You can get some median cost /tok for a given model size. Then make some educated guesses on SotA model sizes, and you can get an estimate on pure cost of serving a model. Error bars and all that, of course. But still a range, with some limits.
No, you can't really make educated guesses unless people start opening their books. Especially in an industry where the vast majority of firms make up valuations out of thin air and not based on any reproducible insights.
Opening their books would let you know things like profitability. I'm talking about cost per token, model development and human costs being irrelevant.
Re: Tokenomics: Quantifying Where Tokens Are Used in Agentic Software Engineering
#77I wrote a Subsack post on this topic back in December https://open.substack.com/pub/zacharywhitley/p/the-coming-ag...
Re: Tokenomics: Quantifying Where Tokens Are Used in Agentic Software Engineering
#78I have a MA system setup for personal use. You give it a problem, you then refine that problem where a fast, cheaper model asks you questions which you answer to get a better input prompt. You then choose a MA strategy for example take problem break up to sections then final judge concludes or you do multi turn where agents debate then judge summarises debate. The best approach is what I call 'all angles' where all t…
Right here is the video demo of what I built - https://streamable.com/e49cgt
Re: Tokenomics: Quantifying Where Tokens Are Used in Agentic Software Engineering
#79Earlier quoted context omitted.
The target codebase is very large. A million tokens is a drop in the proverbial bucket. I still don't understand how caching helps me very much. I must be misunderstanding it because I thought the user's prompt (which is the biggest variable) necessarily sits prior to all of these token intensive tool calls. How can we cache the reading of codebase if the prefix is always moving?
Often to me it seams like using MA is like letting a million monkeys lose. Has ai forgotten about high level design? Surely all it needs to know is what the methods, objects or functions in the code base actually does and the actual code it is meant to be fixing? I wonder if half the issues is that the LLM try to change too much?
[dead]
Re: Tokenomics: Quantifying Where Tokens Are Used in Agentic Software Engineering
#80[flagged]