gaming the metrics instead of using the tools well. this'll show up in the capex ROI question at the next earnings call
Amazon employees are "tokenmaxxing" due to pressure to use AI tools
251–260 of 268 posts
Re: Amazon employees are "tokenmaxxing" due to pressure to use AI tools
#252The fact that management signed off on measuring AI use through token usage shows how incompetent management really is, including in allegedly technical conmpanies like Amazon. Tokenmaxxing was an entirely expected and rational response. IOW You measure employees in stupid ways, you're going to get stupid behaviour as a consequence.
One argument I have heard in favour of this is that management knew this would be a side effect, but that it's more important to have people engage with AI as much as possible simply to explore what is actually possible. You are effectively knowingly wasting money in the expectation that you might learn something useful that will be more valuable in the long run.
in short: dogfood the tool extra hard, and figure out what it can do.
this means wasting a lot of metaphorical dog food, but now everyone will be 100% how it tastes.
this then allows you to shift client expectations and alter offerings.
...
or it's just dumb mgmt. but let's be charitable.
Re: Amazon employees are "tokenmaxxing" due to pressure to use AI tools
#253The fact that management signed off on measuring AI use through token usage shows how incompetent management really is, including in allegedly technical conmpanies like Amazon. Tokenmaxxing was an entirely expected and rational response. IOW You measure employees in stupid ways, you're going to get stupid behaviour as a consequence.
> IOW You measure employees in stupid ways, you're going to get stupid behaviour as a consequence.
My favorite hilarious metric is measuring the amount of work done by counting lines of code written per day
Re: Amazon employees are "tokenmaxxing" due to pressure to use AI tools
#254The fact that management signed off on measuring AI use through token usage shows how incompetent management really is, including in allegedly technical conmpanies like Amazon. Tokenmaxxing was an entirely expected and rational response. IOW You measure employees in stupid ways, you're going to get stupid behaviour as a consequence.
> IOW You measure employees in stupid ways, you're going to get stupid behaviour as a consequence. My favorite hilarious metric is measuring the amount of work done by counting lines of code written per day
> My favorite hilarious metric is measuring the amount of work done by counting lines of code written per day
Or by hours spent in the office
Re: Amazon employees are "tokenmaxxing" due to pressure to use AI tools
#255Earlier quoted context omitted.
How you burn 300 requests in a day? From my Copilot usage Opus consumes surprisingly few requests to do a lot of stuff. It isn’t paying by token but instead by prompt or something.
300 prompts in a day isn't that unreasonable to achieve on a heavy day? And Opus has a significant multiplier as well
That is a lot of prompts. What kind of prompts do you usually give?
Mine are usually giving specification and telling Claude to implement some part of it, referring to existing code base, writing unittests and running e2e until it passes. This can easily take 4-5 hours.
Then again, I have seen colleagues prompt “are you sure?” And other nonsense like that
Re: Amazon employees are "tokenmaxxing" due to pressure to use AI tools
#256Earlier quoted context omitted.
How you burn 300 requests in a day? From my Copilot usage Opus consumes surprisingly few requests to do a lot of stuff. It isn’t paying by token but instead by prompt or something.
I guess you need automation for that. Run claude with cron to find fulnerabilities, suggest and implement improvements, automatically dig through backlog
Hope people doing those kinds of automations are paid to waste prompts and tokens. Any cron based LLM run is just stupid waste
Re: Amazon employees are "tokenmaxxing" due to pressure to use AI tools
#257Earlier quoted context omitted.
How you burn 300 requests in a day? From my Copilot usage Opus consumes surprisingly few requests to do a lot of stuff. It isn’t paying by token but instead by prompt or something.
Review-fix rounds after generation of text or code, until convergence to a solution that doesn't need more improvements.
You are prompting the same model to review what it just did?
Re: Amazon employees are "tokenmaxxing" due to pressure to use AI tools
#258I, too, can easily use more tokens to achieve the same task. I can give worse prompts. I can fail to make it clear to the tools where to find the information they need. I can ask them to think hard when the don’t need to ask tell them not to think when they do need to. I can give vague, open ended instructions. I can generate code that sucks and throw it away. If I do all of this, do I get a promotion?
Even if I'm in the middle of using the AI seriously but then want to rename a variable, I can't do that myself because it'll confuse the AI, so I'll tell it to rename. That seems pretty wasteful.
At least GPT-5.4 and up seem to do okay if I just prefix my next message with something like "(I made edits.)".
Re: Amazon employees are "tokenmaxxing" due to pressure to use AI tools
#259Earlier quoted context omitted.
300 prompts in a day isn't that unreasonable to achieve on a heavy day? And Opus has a significant multiplier as well
That is a lot of prompts. What kind of prompts do you usually give? Mine are usually giving specification and telling Claude to implement some part of it, referring to existing code base, writing unittests and running e2e until it passes. This can easily take 4-5 hours. Then again, I have seen colleagues prompt “are you sure?” And other nonsense like that
I think we use it differently then, I tend to go heavy on the back and forth.
For speccing things out, I have a back-and-forth with the grill-me skill, break things down into tickets, as well as kicking off subagents. That said, I significantly overestimated the number of human messages I send.
My daily 90th percentile wrt number of prompts sent is sitting at 160 queries / day and average at 97 queries / day.
Ran an analysis of my last 2000 messages, with the following breakdown
Task delegation / execution: 23% Investigation / diagnosis / “what’s going on?”: 21% Planning / architecture / brainstorming: 15% Testing / verification / release ops: 10% Review / cleanup / quality control: 9% Course-correction / constraints / preferences: 8% Agent / ticket / workflow orchestration: 6% Providing context / evidence / pasted material: 4% Social reactions / acknowledgements / vibes: 2% Other: 2%
Re: Amazon employees are "tokenmaxxing" due to pressure to use AI tools
#260Old and busted: Hey, whoever amongst you wants to try out AI in the business? Check your idea with legal to make sure we don't get busted for violating HIPAA/GDPR/PCI-DSS and you get 1 k$ worth of tokens at your favorite AI service to play around, if it's any good you get a week extra PTO.
New hotness: USE AI NO MATTER WHAT AND WE WILL MONITOR EVERYTHING, THOSE WHO REFUSE TO USE AI WILL GET FIRED.
And that's how you slop yourself into (at least) two major downtimes and burn millions upon millions of dollars for zero ROI - but the stonk markets don't care about lost ROI as long as you go along with the AI hype train.