Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Is the industry ready for tokens-constrained work?

blog.alaindichiappari.dev

Post reply on HN
Print view View original

51–60 of 78 posts

Re: Is the industry ready for tokens-constrained work?

#51

Post by krab »

sokoloff wrote: post #9

Rationing tokens on a spend level or count makes no sense to me. Why would I pay $10-20K/mo to employ a software engineer and then balk at a $500, $1000, or even $2000 monthly AI bill, assuming they were even vaguely trying to use the tokens productively? I’m not an AI-maximalist, but “work a few days with AI and the rest of the month without because of cost” sounds literally crazy to me. (If you think AI is low/zero…

Because there are diminishing returns somewhere for sure. Will the developer be more effective when he gets $2,000 more in tokens? I can't max the $100 subscription. The same for any other employee-related expense. It's a no-brainer to give the developer a solid laptop. But if you'd pay extra $2,000/month in computer and office supplies costs, will it save the employee a few hours? I don't think so. Their salary doesn't matter that much.

Re: Is the industry ready for tokens-constrained work?

#52

Post by yladiz »

deadbabe wrote: post #14
alainrk wrote: post #10

Earlier quoted context omitted.

Yeah but energy is still a cost, and local inference without batch and multiplexing for many users (like an office would be) is even less optimal. I'm not sure I think it's just pushing the problem forward

Then there will also be a push for cheaper, freer energy as well.

Yeah, with all the AI needs we’ll just have to transition to fusion energy.

Re: Is the industry ready for tokens-constrained work?

#53

Post by loki-ai »

iugtmkbdfil834 wrote: post #36
loki-ai wrote: post #24

Earlier quoted context omitted.

this not making any sense to you is exactly why companies are implementing these caps: you lack the business acumen to understand how financial resources should actually be deployed.

Please consider rewriting this in a way that does not seem to unnecessarily attack parent. You are not wrong about the incentives, but you may not be right about the execution of the restrictions. I should know. I am living through it.

you're fully correct and I apologize for my tone. similarly to many here, I got hit by the caps and frustrated by how many times coworkers were unintentionally or not burning through these resources.

Claiming you have no idea why the managers had to do this felt to me like avoiding any responsibility

Re: Is the industry ready for tokens-constrained work?

#54

Post by sarchertech »

iugtmkbdfil834 wrote: post #29
Avicebron wrote: post #19

Earlier quoted context omitted.

I think in this future they don't think a software engineer is making that much, maybe a nice 50k a year, 65K after 10+ years on the job

I honestly don't know how 100k is not effectively the new 40k from late 90s atm in US. 50K is barely enough to do anything.

I mean median household income in 1999 was $40k and median household income now is $84k.

Re: Is the industry ready for tokens-constrained work?

#55

Post by pdhborges »

jdrek1 wrote: post #28
pdhborges wrote: post #13

Earlier quoted context omitted.

10 grand a month for a junior is absolutely ridiculous in most of the world including large swats of the western world.

They were talking about what it costs the company, not just the salary. There are taxes, insurances, equipment costs, etc. 10k might be a bit too high, but it's far from "absolutely ridiculous" amounts of being too high. An employee with a 5k salary can easily cost the employer 7-8k

I took into account all of that. Still ridiculous.

Re: Is the industry ready for tokens-constrained work?

#56

Post by pdhborges »

Kirby64 wrote: post #32
pdhborges wrote: post #13

Earlier quoted context omitted.

10 grand a month for a junior is absolutely ridiculous in most of the world including large swats of the western world.

Fully loaded employee cost is typically around 2x their actual salary. Junior employees making only $60k/yr is absolutely nothing. It’s lowish even in most parts of the western world.

You have people in Spain and Portugal hiring juniors at 25k per year.

Re: Is the industry ready for tokens-constrained work?

#57

Post by jgmedr »

Currently navigating this situation at my employer. We went from virtually unlimited token spend per software engineer, to $150 per month due the recent change in billing terms from GitHub Copilot. Rationed over a month, about $7.50 per day. Basically, a couple bad apples spoiled the bunch (contractors using Opus to center divs).

We're 15 days into this new policy and its going ~okay~. Engineers adapt as they do, and have been leaning on `gpt-5.6-luna xhigh`. Some contractors have already hit their budget limit for the month.

A couple observations here:

1. Because LLMs/agents are tools, limiting their usage becomes a distraction and ends up being more of a drag on each individual's productivity. Instead of "just doing the work" engineers are now wasting time tinkering with setups (graphs, caveman skills, etc.).

2. Skill atrophy is real. Engineers that hit their budgets are seemingly less productive and less capable which is deeply concerning.

3. There are legitimate conversations happening now to explore open source harnesses (opencode/pi) and open weight models at the company in order offset the costs associated with going through a standard provider.

4. Token prices are venture capital subsidies. Its essentially free samples to get the market hooked on their addictive white collar drug. Remember when an Uber cost $7? As soon as OpenAI and Anthropic go public, they will need to begin showing progress toward profitability. That is when the true price of a token will be revealed.

Re: Is the industry ready for tokens-constrained work?

#58

Post by sokoloff »

loki-ai wrote: post #53
iugtmkbdfil834 wrote: post #36

Earlier quoted context omitted.

Please consider rewriting this in a way that does not seem to unnecessarily attack parent. You are not wrong about the incentives, but you may not be right about the execution of the restrictions. I should know. I am living through it.

you're fully correct and I apologize for my tone. similarly to many here, I got hit by the caps and frustrated by how many times coworkers were unintentionally or not burning through these resources. Claiming you have no idea why the managers had to do this felt to me like avoiding any responsibility

I am the manager in this case and the one who sets the AI policy for my low-8-figure/year product development teams.

Our policy is quite permissive for exactly the reasons I conveyed above. It takes an extremely finely-tuned sense of value and high assurance that you’re on the diminishing return portion of the curve to conclude “SWEs should have this precisely metered amount (rather than zero or ‘as much as they don’t waste’)”

My claiming to have no idea was not an abdication of responsibility but rather a claim that it was likely an error (assuming a non-trivial team size and a business that can continue to grow).

Re: Is the industry ready for tokens-constrained work?

#59

Post by Ekaros »

deadbabe wrote: post #21
alainrk wrote: post #16

Earlier quoted context omitted.

Right, every technology is suboptimal in the beginning, but constraints will ever exists, energy and hardware in primis. Aldo you have to factor entropy in regardless, complexity remains a cost

Imagine if people had said one day computer programs had to be more memory constrained because everything uses too much RAM. Madness. We'd never see 128GB of RAM.

We would be lot better off. I am very weirded that computers are one of the place where waste is sometimes even celebrated. We would be laughed out if we suggested same with say energy in general. Why have walls, we could just have enough space heaters to always push warm air or cool air... Saying we need walls stops us reaching point where we heat or cool ourselves in outdoors constantly.

Re: Is the industry ready for tokens-constrained work?

#60

Post by cebert »

the__alchemist wrote: post #17
cebert wrote: post #2

I have similar concerns about this. The past year or so, software engineers have been encouraged by employers to adopt AI tooling and agentic coding. Now, many enterprises, including mine, are starting to crack down on token spend. I’ve learned how to fully embrace agentic tooling to do tasks like keeping up with vulnerability reports, initially triaging defects that come in, etc. It would be hard for me to do things…

Maybe a dumb question, but what are you doing/seeing others do that uses that many tokens? I find myself pressing the weekly caps on the $20/month plan only when I'm really having the LLMs go wild with the Xtra high effort on the newest/biggest models on abstract problems or where I don't really understand the problem well. (E.g. improving ML models, planning creative molecule synthesis pipelines, identifying subtle…

> Maybe a dumb question, but what are you doing/seeing others do that uses that many tokens? I find myself pressing the weekly caps on the $20/month plan

I don’t run into many issue with my personal subscription plans. However, at work while companies like Anthropic offer “Enterprise” plans, you still get billed at API rates which are quite high. The subscription plans are an incredible deal and subsidize these tokens, but these plans aren’t offered to large organizations.

Post reply on HN