Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Is the industry ready for tokens-constrained work?

blog.alaindichiappari.dev

Post reply on HN
Print view View original

21–30 of 78 posts

Re: Is the industry ready for tokens-constrained work?

#21

Post by deadbabe »

alainrk wrote: post #16
deadbabe wrote: post #12

I don't think people understand, tokens must never be constrained. Right now, LLMs are the worst they'll ever be, but imagine what they will be like at the peak: Anything you want to code, coded instantly. Not waiting for code to stream in or wait hours for some agents to crunch through loops and planned: it just appears on the screen instantly like the way a webpage loads. Then imagine it can be done locally, on you…

Right, every technology is suboptimal in the beginning, but constraints will ever exists, energy and hardware in primis. Aldo you have to factor entropy in regardless, complexity remains a cost

Imagine if people had said one day computer programs had to be more memory constrained because everything uses too much RAM. Madness. We'd never see 128GB of RAM.

Re: Is the industry ready for tokens-constrained work?

#22

Post by neaden »

pdhborges wrote: post #13
neaden wrote: post #6

Earlier quoted context omitted.

It's also kind of odd because even a junior engineer has to cost a company around 10 grand a month in salary, taxes, and other costs. So if the company thinks it makes you 10% more effective it seems like it should be an easy choice. So either the companies are shooting themselves in the foot limiting spending or they aren't seeing the productivity boost.

10 grand a month for a junior is absolutely ridiculous in most of the world including large swats of the western world.

Between all the benefits, taxes, and the infrastructure the company needs costs rise pretty quickly.

Re: Is the industry ready for tokens-constrained work?

#23

Post by monkeydust »

cebert wrote: post #2

I have similar concerns about this. The past year or so, software engineers have been encouraged by employers to adopt AI tooling and agentic coding. Now, many enterprises, including mine, are starting to crack down on token spend. I’ve learned how to fully embrace agentic tooling to do tasks like keeping up with vulnerability reports, initially triaging defects that come in, etc. It would be hard for me to do things…

On business side orgs are now absorbing costs of tokens to their own local budgets...so the calculus soon will be:1 extra engineer or extra token spend for existing ones. Tbh, when the productivity payoff becomes more obvious I would happily take the latter.

Re: Is the industry ready for tokens-constrained work?

#24

Post by loki-ai »

sokoloff wrote: post #9

Rationing tokens on a spend level or count makes no sense to me. Why would I pay $10-20K/mo to employ a software engineer and then balk at a $500, $1000, or even $2000 monthly AI bill, assuming they were even vaguely trying to use the tokens productively? I’m not an AI-maximalist, but “work a few days with AI and the rest of the month without because of cost” sounds literally crazy to me. (If you think AI is low/zero…

this not making any sense to you is exactly why companies are implementing these caps: you lack the business acumen to understand how financial resources should actually be deployed.

Re: Is the industry ready for tokens-constrained work?

#26

Post by unified101 »

the__alchemist wrote: post #17
cebert wrote: post #2

I have similar concerns about this. The past year or so, software engineers have been encouraged by employers to adopt AI tooling and agentic coding. Now, many enterprises, including mine, are starting to crack down on token spend. I’ve learned how to fully embrace agentic tooling to do tasks like keeping up with vulnerability reports, initially triaging defects that come in, etc. It would be hard for me to do things…

Maybe a dumb question, but what are you doing/seeing others do that uses that many tokens? I find myself pressing the weekly caps on the $20/month plan only when I'm really having the LLMs go wild with the Xtra high effort on the newest/biggest models on abstract problems or where I don't really understand the problem well. (E.g. improving ML models, planning creative molecule synthesis pipelines, identifying subtle…

The retail account is about 200 to 300 dollars of api equivalent. Large companies disallow retail.

Re: Is the industry ready for tokens-constrained work?

#27

Post by abuani »

sokoloff wrote: post #9

Rationing tokens on a spend level or count makes no sense to me. Why would I pay $10-20K/mo to employ a software engineer and then balk at a $500, $1000, or even $2000 monthly AI bill, assuming they were even vaguely trying to use the tokens productively? I’m not an AI-maximalist, but “work a few days with AI and the rest of the month without because of cost” sounds literally crazy to me. (If you think AI is low/zero…

What about a $10k to $20k bill? Or better, 100%-200% of the employees salary? At a certain point in time, finance is going to care greatly about the ability to predict costs. Pre AI, predicting the cost an engineer was quite easy. Salary + benefits + licenses for software. CI systems that were unbounded were rolled up into opex and treated as a separate line item. Where does token spend land in this now?

Finance teams above all else value predictability. Token spend is the opposite of predictable. Eventually these two forces are going to collide and some part of the system will break. My guess is it's token spend, not finance allowing unbound spend on a previously predictable part of the balance sheet.

Re: Is the industry ready for tokens-constrained work?

#28

Post by jdrek1 »

pdhborges wrote: post #13
neaden wrote: post #6

Earlier quoted context omitted.

It's also kind of odd because even a junior engineer has to cost a company around 10 grand a month in salary, taxes, and other costs. So if the company thinks it makes you 10% more effective it seems like it should be an easy choice. So either the companies are shooting themselves in the foot limiting spending or they aren't seeing the productivity boost.

10 grand a month for a junior is absolutely ridiculous in most of the world including large swats of the western world.

They were talking about what it costs the company, not just the salary. There are taxes, insurances, equipment costs, etc.

10k might be a bit too high, but it's far from "absolutely ridiculous" amounts of being too high. An employee with a 5k salary can easily cost the employer 7-8k

Re: Is the industry ready for tokens-constrained work?

#29

Post by iugtmkbdfil834 »

Avicebron wrote: post #19
sokoloff wrote: post #9

Rationing tokens on a spend level or count makes no sense to me. Why would I pay $10-20K/mo to employ a software engineer and then balk at a $500, $1000, or even $2000 monthly AI bill, assuming they were even vaguely trying to use the tokens productively? I’m not an AI-maximalist, but “work a few days with AI and the rest of the month without because of cost” sounds literally crazy to me. (If you think AI is low/zero…

I think in this future they don't think a software engineer is making that much, maybe a nice 50k a year, 65K after 10+ years on the job

I honestly don't know how 100k is not effectively the new 40k from late 90s atm in US. 50K is barely enough to do anything.

Re: Is the industry ready for tokens-constrained work?

#30

Post by izacus »

sokoloff wrote: post #9

Rationing tokens on a spend level or count makes no sense to me. Why would I pay $10-20K/mo to employ a software engineer and then balk at a $500, $1000, or even $2000 monthly AI bill, assuming they were even vaguely trying to use the tokens productively? I’m not an AI-maximalist, but “work a few days with AI and the rest of the month without because of cost” sounds literally crazy to me. (If you think AI is low/zero…

> Why would I pay $10-20K/mo to employ a software engineer and then balk at a $500, $1000, or even $2000 monthly AI bill, assuming they were even vaguely trying to use the tokens productively?

No idea, but companies refusing to buy a 2000$ laptop once every 3 years, a new mouse, keyboard, or screen for So I'd expect those to not suddenly change into "let's waste money" companies.

Post reply on HN