Managing AI Coding Costs at Scale • Hacker News Board

Re: Managing AI Coding Costs at Scale

#221

Post by Footprint0521 » Sat, Aug 08, 2026, 11:18 AM UTC

blcknight wrote: post #134

extr wrote: post #16

I would be really curious to hear from devs at Databricks what the experience of development is like internally. I work at a small startup with essentially unlimited AI spend budget - the entire point is that I should be turning to it at every opportunity since our human labor is so expensive relative to tokens. So generally it's like: - Spend most time prioritizing/discussing what to do. - Once that's agreed, use Fa…

$80 sounds extremely low for what you're describing - are you on API token plans? I have had some $3,000 token days - even without Fable. I don't see how this is sustainable. My personal 20x plans get so much usage for so cheap. The consumer subsidies are crazy, but alas I can't use them for work.

Dude $3k? Holy heck you should look into K3/Deepseek V4 Flash

Re: Managing AI Coding Costs at Scale

#222

Post by antonvs » Sat, Aug 08, 2026, 11:23 AM UTC

thereitgoes456 wrote: post #202

gizmodo59 wrote: post #146

Earlier quoted context omitted.

I will believe there is no moat when the revenues for Anthropic is not 70B. It seems like people want to throw away money and they don’t like switching

There is no evidence that Anthropic's revenue is 70B.

https://www.linkedin.com/posts/sytaylor_anthropic-is-now-tra...

Re: Managing AI Coding Costs at Scale

#223

Post by Footprint0521 » Sat, Aug 08, 2026, 11:26 AM UTC

platinumrad wrote: post #3

Careful. If you admit to using models that weren't trained by OpenAI or Anthropic then you might hauled in front of Congress: https://www.scmp.com/news/china/diplomacy/article/3362616/us...

That is a wild article! Hopefully the recent US open source model initiatives can pan out to cut costs, because I’m not a fan of dependence on Chinese models either, but I’m in a spot where I need to maximize intelligence on minimal cost…

There’s not even a distant comparison US lab rn which is sad

Re: Managing AI Coding Costs at Scale

#224

Post by dan_q » Sat, Aug 08, 2026, 11:27 AM UTC

lbriner wrote: post #17

There are a surprising number of articles like this along the lines of, "we started using AI tools and ended up spending millions per year". On what planet do people start paying for things without keeping an eye on the costs and no-one notices until you have spent a crazy amount? I don't understand. You are either paying a fixed amount which you are happy about in-advance or you are PAYG in which case you would ball…

> On what planet do people start paying for things without keeping an eye on the costs and no-one notices until you have spent a crazy amount? I don't understand. You are either paying a fixed amount which you are happy about in-advance or you are PAYG in which case you would ballpark how much it costs.

The AI rollout has in large part been about giving management an opportunity to say "fuck you" to engineering.

Engineering payroll needed to be slashed and the C-Suite was happy to let their managers get some lashes in on the uppity workers.

The point of AI is:

1. It's bad engineering and everyone knows it.

2. Management is pushing it through anyway.

3. Deal with it, asshole.

The ROI is besides the point. It's about breaking the spirit of engineering labor with irrational decisions made outside of their control.

Edit: It is just as much about affirming the exalted status of management in comparison to labor. It's a bitter reinscription of the status hierarchy in tech; Management is good and labor is bad; Management is considerate and labor is ignorant; Management is strong and labor is weak; Management is superior and labor is inferior.

Re: Managing AI Coding Costs at Scale

#225

Post by newsicanuse » Sat, Aug 08, 2026, 11:41 AM UTC

extr wrote: post #16

I would be really curious to hear from devs at Databricks what the experience of development is like internally. I work at a small startup with essentially unlimited AI spend budget - the entire point is that I should be turning to it at every opportunity since our human labor is so expensive relative to tokens. So generally it's like: - Spend most time prioritizing/discussing what to do. - Once that's agreed, use Fa…

Makes me wonder the kind of startup this peron is working for where slop is encouraged

Re: Managing AI Coding Costs at Scale

#226

Post by skydhash » Sat, Aug 08, 2026, 11:56 AM UTC

williamstein wrote: post #196

thelastgallon wrote: post #182

> nearly every company deploying AI tools at scale has hit the same wall: exponentially growing costs. That curve is unsustainable - left unchecked it will eventually overtake revenue. But the question remains, AI hasn't shown any explosion in growth/revenue for most companies. The effects seem muted.

If only one company had AI and nobody else did, then that one company would be expected to start seeing an explosion in growth/revenue. As it is, everybody has access to AI, so extensive use of AI should just result in keeping up with the competition. The ROI to spending on AI is "not dying".

AI is sold as a power tool. Even if everyone is using it, you should see at least some absolute improvement in boost, even though they are still relatively the same. No such boost has been observed.

Re: Managing AI Coding Costs at Scale

#227

Post by xbmcuser » Sat, Aug 08, 2026, 12:08 PM UTC

pcox wrote: post #191

pianopatrick wrote: post #165

Earlier quoted context omitted.

I think part of the reason software was winner take most was the difficulty of making software. I remember hearing a story that in the past movies were so technically difficult to make that any movie that got made had a good chance to be a profitable hit. But as movies got cheaper to make, more movies got made. Nowadays movie studio execs have to really calculate out the audience and expected revenue for any new movi…

> I think part of the reason software was winner take most was the difficulty of making software. That might be part of it, but I think it also has to do with the reality of replicating and scaling. Hardware or physical goods simply don’t scale like digital goods. There can be hundreds of knock-off physical products that have lower quality and lower cost but serve 90% of the same purpose, because physical capacity fo…

I disagree the worth of software will become a lot less as LLM take hold and more people outside software devs know how to use them properly. I am the default IT manager for our small office as I am the guy most interested in tech and in the last 2-3 years I am getting llm to write python scripts for me to do work that would have cost $100s of dollars. Currently you need to pay $100-1000s for a software with 100s of features while you might actually need 1 or 2 of the features which you can get the llm to write for you.

As the LLM's keep getting better at writing software actual coding skills required will decrease which will result in a huge influx of software crashing prices. You can see this in already in open source projects in the last 6 months to a year Jellyfin had a huge influx of custom clients made by different people for fixing their own itches about the official software.

Re: Managing AI Coding Costs at Scale

#228

Post by imilev » Sat, Aug 08, 2026, 12:18 PM UTC

extr wrote: post #16

I would be really curious to hear from devs at Databricks what the experience of development is like internally. I work at a small startup with essentially unlimited AI spend budget - the entire point is that I should be turning to it at every opportunity since our human labor is so expensive relative to tokens. So generally it's like: - Spend most time prioritizing/discussing what to do. - Once that's agreed, use Fa…

Interested to dive deeper on the upfront design discussion. Have you found these to more often then not translate into the real product.

In my experience at the begining of the full agentic coding loop in our company we were more hands on with the codebase and had better judgement over the plans. Now it is quite often that the inital plan after executed needs more refinement and that made the plan review somewhat obsolete for us.

Re: Managing AI Coding Costs at Scale

#229

Post by imilev » Sat, Aug 08, 2026, 12:20 PM UTC

Great article, I think the most important aspect from it is the auto-routing. As humans laziness is in our nature, so having to think if the model is capable enough is not something that most ppl will do - resulting in trying out smaller models which failed our task and then just giving up and running on the bigger model all the time.

Re: Managing AI Coding Costs at Scale

#230

Post by lazarie » Sat, Aug 08, 2026, 12:20 PM UTC

sashank_1509 wrote: post #173

I suspect that when it comes to hard complex software products, you’re better off ignoring agents and doing “trad coding”. What you lose in short term speed you gain in manageable complex codebases. If you have a 500k line codebase and even > 50% is written by agents, you are in a world of pain that won’t justify the costs longer term. Now of course, there are products that just involve lots of code but are not actua…

I work on and maintain a 900k+ LoC codebase with mixed languages, runtimes, deployment models, cloud and airgapped ops and it's 99% written by AI. The difference I see the most between our approach and others is that we see the end-user product as an output of our context engineering internal machine, we focus on building that machine and we measure the output like madmen across all and any qualitative software metrics and signals (traces,collectors,otel,snapshots,profilers,custom debuggers,videos and more). For any change we do, less than 5% of wall time is an AI doing reasoning or coding, 95% is running the verification deterministically. Of that 5%, about 90% is spent on non-feature aligned code.

Velocity and quality is through the roof, "10x" would be a regression at this point, the bottleneck is only figuring out what is relevant/good to deliver, the implementation is the output of the machine.