Live data from Hacker News

Managing AI Coding Costs at Scale

databricks.com

91–100 of 267 posts

Re: Managing AI Coding Costs at Scale

#91
Surprisingly pragmatic and info packed article..

Kudos to databricks, I also find it interesting that such different companies (Stripe, Ramp, Databricks) are all building the exact same internal tools.

I think building companies is going to look more generic in the future because intelligence is an API now.

Re: Managing AI Coding Costs at Scale

#92

This approach seems fundamentally predicated on being able to evaluate coding agents on your own code by having domain specific evals. With that knowledge, you can trust the routing logic is actually improving/maintaining perf while reducing costs. Without the insight into agent performance, any changes like this feel like a gamble to save $$ at the cost of developer productivity I'm actually working on building gene…

The difficulty of evaluating coding agents is indeed a really big challenge. We built evals on our own codebase and shared some information about that to allow other companies to replicate. We found our own evals correlated loosely with public generic SWE benchmarks.

In large user populations like at Databricks I think the ultimate answer will come from experimentation instead of offline evals. We are already doing this in small groups, exposing them to new candidate models and then measuring per-developer cost and perceived quality changes.

Re: Managing AI Coding Costs at Scale

#93
So did we. I just asked my team to get personal accounts that I reimburse them for. It’s just a golden age loop though, the gravy train can’t go on forever unless we start building out thousands of data centers and associated renewable energy.

Re: Managing AI Coding Costs at Scale

#94

First time hearing of Omnigent. Anyone have experience using it?

I've been using it for a week or so. The main draw for me is that I can keep my sessions in one database regardless of the model/provider I use. The webapp can access everything remotely, which is convenient when I'm on my phone.

I haven't gotten a chance to test the multi-agent capabilities, but the DeepSeek Flash prices are so low that I probably will soon.

Re: Managing AI Coding Costs at Scale

#95
post #50
post #37

Earlier quoted context omitted.

This was more true a few months ago but Fable has improved the situation considerably. Also just remember - minimalist code looks and feels great but customers do not read your code. I have caught myself many times providing "corrections" to abstractions that were already ~fine, just not perfect. The average SWE costs $200/hr. Careful you don't burn $50 worrying about code that will likely be rewritten or can be bett…

> The average SWE costs $200/hr. And this is how I find out I'm woefully underpaid.

Whatever you are making this year as SWE you'll be making less next year if the current trend in improvement of AI coding aids is going to be sustained. Think about it : programmers used to derive a lot of their value from the fact that it was a hard skill to acquire. My kids can now 'vibe code' stuff faster (and better looking) than what I could come up with as the beginnings of a design plan. And then I still need to implement it.

Re: Managing AI Coding Costs at Scale

#96

Surprisingly pragmatic and info packed article.. Kudos to databricks, I also find it interesting that such different companies (Stripe, Ramp, Databricks) are all building the exact same internal tools. I think building companies is going to look more generic in the future because intelligence is an API now.

Thank you for the feedback. We wrote this because after discussing with some of our peer companies, I realized everyone was roughly doing similar things. And I thought it would be good for someone to just systematically write down what those are so that others can try out the techniques if they find them useful.

Re: Managing AI Coding Costs at Scale

#97
post #36

Earlier quoted context omitted.

> I work at a small startup How does a “small startup” end up with a multi million line “legacy” codebase? Something not mathing

> How does a “small startup” end up with a multi million line “legacy” codebase? Easy! The output of 6 months ago Opus! Which seemed so wonderful at the time.

This! I don't think folks understand how easy it is to go from greenfield to brownfield with these tools, esp if your organization is only valuing velocity. Meaning your doing full agentic development on large features, barely reviewing any code, and shipping without much refinement. It's insane, but this appears to be the status quo in SF startups.

Re: Managing AI Coding Costs at Scale

#99
post #49

Earlier quoted context omitted.

How is this not true? Taking a Senior SWE @ ~$200K, even just the base salary cost / 2080 working hours is $100/hr. Fully loaded employer cost + accounting for non-coding time gets you to upper 100s easily. Even for a junior making $100K, I have a hard time believe their time is worth less than $75/hr or so. Edit: Fine, "Senior" is not "Average". But naive salary is not the true numerator.

I have a hard time believe their time is worth less than $75/hr or so. In many places in Europe it is.

Western Europe is mostly consultancy, and the rate paid by client is usually higher, and doesn't matter if it's eastern Europe, Portugal or even India.

Re: Managing AI Coding Costs at Scale

#100
post #61

Earlier quoted context omitted.

Something isn't clear about the size of your codebase here and the level of reliability your customers expect, as a reader of your comments. Clarity there will help. My observation has been: - Initial greenfield work by an LLM is fast and very effective with minimal or no human oversight. - Subsequent work ends up being over engineered and very verbose. Assumptions are made that aren't suited to the problem at hand (…

> unguided LLM usage Why aren't you guiding your LLM usage? Is that what I said - to spam it and not guide anything? Or to have a careful workflow where you agree on design and maximize your human judgement/leverage? > any state that's not explicitly being tested and verified in QA loops As opposed to before, when engineers perfectly reasoned about code behavior from first principals and QA was unnecessary?

AI is an accelerate tool for any organizations, management thinks it'll solve their organization issue because it accelerates it. Most often, it accelerates toward a wall.

Design is too expensive, we do agile. QA too expensive, we fire all of them, and claim devops is the now, which allows us to fire the Ops team too, 100% ownership from deisng to ops on devs.

One person with an agent can replace all these teams. Yeah mo profits.

Post reply on HN