I think the biggest problem is not necessarily the cost to develop & serve the models, but how quickly user behavior changed with token based pricing. I know a lot of people at companies where the marching orders changed on a dime end of Q1/start of Q2. These are shops that were fully on the "use AI or die (because we will fire you)" train. Now there's monitoring, reporting, alerting not just on overall cost but on "…
I can give you some additional anecdotal evidence to support your comment. I work at a Fortune 200 company. At first, it was the Wild West. Need an LLM? You got it. Need to or want to build an army of agents? Done and done. We literally had everything at the tips of fingers for about 3 months. Teams were building their own internal tools, the team I work on canceled contracts with several software vendors because tea…
I think the biggest problem is not necessarily the cost to develop & serve the models, but how quickly user behavior changed with token based pricing. I know a lot of people at companies where the marching orders changed on a dime end of Q1/start of Q2. These are shops that were fully on the "use AI or die (because we will fire you)" train. Now there's monitoring, reporting, alerting not just on overall cost but on "…
What I don't understand is why some companies are so stingy on AI token usage. If you're paying an engineer $X and they're getting 3x the amount of work done you should be happy paying up to $2X in AI tool usage. In reality many companies start complaining at their employees when they hit $0.1*X or less.
10% of a developer's cost is something like $4000/month. Many companies are complaining at a point that's well, well below that.
(I think they are being irrational, and that the mental model they have of AI costs -- "how much are we spending on tooling for this developer?" -- is going to shift over time to something more sensible, but those kinds of short-sighted companies are the ones that are having cost panics.)
It's not an affordability crisis, it's a financial crisis. The models get cheaper super fast. By this time next year Fable 5 will cost less than Sonnet does today. That's not the problem. The problem is that many companies are going to realize that they don't get any ROI from AI. Generating code faster != more profit. Most of the Fortune 500 will likely realize this and then the token budgets will come crashing down.…
Do they get ROI from additional workers? Honestly it is difficult for me to imagine white collar companies that have positive marginal profit of labor but not positive marginal profit of tokens, especially in the future.
Considering the massive layoffs and cold market post COVID, the answer is "no" for a lot of companies.
Theoretically, there's a lot of room for marginal work where developer time isn't worth the cost for the output but tokens are cheap enough to make it worthwhile. Very little of that work ends up being customer facing though, so it isn't actually a growth opportunity for the company.
I think the biggest problem is not necessarily the cost to develop & serve the models, but how quickly user behavior changed with token based pricing. I know a lot of people at companies where the marching orders changed on a dime end of Q1/start of Q2. These are shops that were fully on the "use AI or die (because we will fire you)" train. Now there's monitoring, reporting, alerting not just on overall cost but on "…
What I don't understand is why some companies are so stingy on AI token usage. If you're paying an engineer $X and they're getting 3x the amount of work done you should be happy paying up to $2X in AI tool usage. In reality many companies start complaining at their employees when they hit $0.1*X or less.
I hesitate to say this because I think the AI hype is generally overdone, but I was contemplating the other day how much I would recommend my employer spend on AI tooling for me based on my salary and it's got to be in the tens of thousands of dollars per year if I'm being honest. I'm a contractor. On my most recent client, I was able to learn a completely new domain and business, design a solution and build a full stack POC (a complicated one with technologies I had never used before) nearly singlehandedly in less than 2 months. That includes all of the onboarding time, getting access to the right people, repositories, databases etc. What made it possible was 1) the AI helped me understand technologies I had never used before, from a 30 year old Java stack to graph databases, 2) it could analyze DDLs and make correct inferences about what tables and columns meant in both business and technical terms, 3) it generated mostly-correct code in multiple languages to achieve what I needed it to do, and 4) it referenced libraries I didn't know existed to accomplish tasks I needed done. I could cycle between these activities and iterate FAR more rapidly than I ever have before. To me this is the thing the AI companies should really
be emphasizing. I'm not convinced autonomous agents are really working out, but I'm 1000% sold on their ability to empower developers. And I say this as a pretty skeptical late adopter who only knows the most basic AI tooling.
It’s a massive supply crunch. More production will come online.
Probably not, at least for DRAM. The demand has historically been very variable, and building production capacity takes multiple years. Also, spare capacity is really expensive. Thus the memory manufacturers don't want to expand, betting on it being yet another temporary bubble. Also, DRAM fabs are not really usable to make compute (CPU, GPU, etc in this context) silicon. The production lines and tech have diverged s…
How did Apple figure this out? Isn't the solution to this to sort of evolve to a unified memory architecture, which wins in speed and cost anyway?
The coming AI enshittification is going to be epic. For those of us who have been on the web for more than five minutes, we can see this a mile away. If you think search ads are annoying, pre-roll YouTube ads are annoying, streaming ads are annoying, or basically ads-on-any-screen-anywhere-at-any-time are annoying, just wait until every stupid thing is powered by AI and is subtly trying to manipulate you to buy/watch…
I'm not sure how that would legally work in EU: several countries have strict rules about ads having to be clearly distinguished from non-ads material. I know that UK has pretty strict rules about product placement in broadcasts too for example.
Yes they can do ads, but if they try to be subtle they will likely (eventually) be hit with fines.
Though, do the current rules apply to AI? Likely unclear. But if this becomes a problem I would expect new consumer protection regulation to be introduced aimed at this specific issue.
What I don't understand is why some companies are so stingy on AI token usage. If you're paying an engineer $X and they're getting 3x the amount of work done you should be happy paying up to $2X in AI tool usage. In reality many companies start complaining at their employees when they hit $0.1*X or less.
Most companies are not seeing 2x or 3x of value produced from developers with AI is the reason
Yeah the cost doesn't justify the value. Spending x3 on tokens is meaningless if they aren't seeing the profit side scale with it. Going faster on product features doesn't mean it translates to more money. In fact I'm not even sure some customers can handle the speed at which a team could move with feature delivery.
So scaling horizontally in different markets seems like an advantage if a product is already mature. Which is exactly what Anthropic and Open AI are doing because they want to put their tentacles in everything.
It's just schadenfreude/ressentiment. Our societal values are very christian even as we secularize and you can see this reflected everywhere you turn. Same thing with people feeling there ought to be a 'catch' to GLP1s, etc. etc.
I don't get you because I'm too stupid to parse your thoughts
I do a lot of client work for fortune 100’s. Over the last month I have seen companies scrambling to measure deliverables against cost. Most of the back room talk is to the affect of giving devs a small allowance ($500 a month) and then making them prove their own productivity increases (again, based on deliverables, not LoC) before they either take it away or give them more. Obviously this won’t be on an individual…
The AI companies will be profitable if they ship the goods. Make AGI and companies will pay for it.
Not if it costs more than a human, or if it performs worse (be that in quality or in time). After all, humans have NGI and companies don't just hire anyone with a pulse.
I think the biggest problem is not necessarily the cost to develop & serve the models, but how quickly user behavior changed with token based pricing. I know a lot of people at companies where the marching orders changed on a dime end of Q1/start of Q2. These are shops that were fully on the "use AI or die (because we will fire you)" train. Now there's monitoring, reporting, alerting not just on overall cost but on "…
This is great to see. LLMs are great but the industry really needs to experience a correction.