Is the industry ready for tokens-constrained work?
blog.alaindichiappari.dev
Is the industry ready for tokens-constrained work?
1–10 of 78 posts
Re: Is the industry ready for tokens-constrained work?
#2I have similar concerns about this. The past year or so, software engineers have been encouraged by employers to adopt AI tooling and agentic coding. Now, many enterprises, including mine, are starting to crack down on token spend.
I’ve learned how to fully embrace agentic tooling to do tasks like keeping up with vulnerability reports, initially triaging defects that come in, etc. It would be hard for me to do things the old way at this point when I know tools are available that could make me more productive for a particular category of tasks.
My employer is considered capping all engineers at $200 or $500/mo of token spend depending on level. I regularly spend over $1k/mo today, but believe I can make a strong business justification for the value those tokens are creating.
At this point, I think engineers may be asking what token budgets are when considering new roles.
Re: Is the industry ready for tokens-constrained work?
#3I have similar concerns about this. The past year or so, software engineers have been encouraged by employers to adopt AI tooling and agentic coding. Now, many enterprises, including mine, are starting to crack down on token spend. I’ve learned how to fully embrace agentic tooling to do tasks like keeping up with vulnerability reports, initially triaging defects that come in, etc. It would be hard for me to do things…
Indeed, but I also think structural considerations must be done at the company level on the work model overall
Re: Is the industry ready for tokens-constrained work?
#4I have similar concerns about this. The past year or so, software engineers have been encouraged by employers to adopt AI tooling and agentic coding. Now, many enterprises, including mine, are starting to crack down on token spend. I’ve learned how to fully embrace agentic tooling to do tasks like keeping up with vulnerability reports, initially triaging defects that come in, etc. It would be hard for me to do things…
I think eventually we will converge on a new AI + old AI (ILP, ASP) way of working.
Re: Is the industry ready for tokens-constrained work?
#5I tend to think this will end up being a good thing. It will force people to actually think about the best use cases and utilization of tokens while others will push to optimize models and tools to improve cost and efficiency. “Life finds a way”
Re: Is the industry ready for tokens-constrained work?
#6I have similar concerns about this. The past year or so, software engineers have been encouraged by employers to adopt AI tooling and agentic coding. Now, many enterprises, including mine, are starting to crack down on token spend. I’ve learned how to fully embrace agentic tooling to do tasks like keeping up with vulnerability reports, initially triaging defects that come in, etc. It would be hard for me to do things…
It's also kind of odd because even a junior engineer has to cost a company around 10 grand a month in salary, taxes, and other costs. So if the company thinks it makes you 10% more effective it seems like it should be an easy choice. So either the companies are shooting themselves in the foot limiting spending or they aren't seeing the productivity boost.
Re: Is the industry ready for tokens-constrained work?
#7You know what happened the last years when the internet went down and there were no emails, no Microsoft Teams, no "npm install", no stackoverflow, no Google, no coordination with other locations?
You know what happened when a PC broke? You know what happens at a construction site when the excavator breaks down?
Exactly nothing — and that's okay.
Re: Is the industry ready for tokens-constrained work?
#8I tend to think this will end up being a good thing. It will force people to actually think about the best use cases and utilization of tokens while others will push to optimize models and tools to improve cost and efficiency. “Life finds a way”
I think it will be a good thing for other reasons: there will be a massive push to get hardware good enough to run local LLMs burning infinite tokens.
Re: Is the industry ready for tokens-constrained work?
#9Rationing tokens on a spend level or count makes no sense to me. Why would I pay $10-20K/mo to employ a software engineer and then balk at a $500, $1000, or even $2000 monthly AI bill, assuming they were even vaguely trying to use the tokens productively?
I’m not an AI-maximalist, but “work a few days with AI and the rest of the month without because of cost” sounds literally crazy to me. (If you think AI is low/zero/negative net value, don’t do the first half; if it has net value, don’t do the second half.)
Re: Is the industry ready for tokens-constrained work?
#10I tend to think this will end up being a good thing. It will force people to actually think about the best use cases and utilization of tokens while others will push to optimize models and tools to improve cost and efficiency. “Life finds a way”
I think it will be a good thing for other reasons: there will be a massive push to get hardware good enough to run local LLMs burning infinite tokens.
Yeah but energy is still a cost, and local inference without batch and multiplexing for many users (like an office would be) is even less optimal.
I'm not sure I think it's just pushing the problem forward