Live data from Hacker News

Managing AI Coding Costs at Scale

databricks.com

121–130 of 269 posts

Re: Managing AI Coding Costs at Scale

#121
post #26

Earlier quoted context omitted.

Do you have issues with performance at the moment? Right now I tend to find that it produces absolutely terrible design patterns and especially performance. I mean maybe I don't know exactly what area you're looking at but yeah for us we tend to find it's terrible wrt dB/caching/scaling and often any performance improvements it proposes end up actually shooting itself in the foot and being worse than before but it's…

Performance is better than ever. It's never been more practical to set up wildly complex synthetic test environments and measure perf wins. Plus the models will find every possible algorithmic/design improvement. It actually gives me quite an uncanny feeling, bulldozing over years of human optimization work with a newer, "perfect" design. Like bringing an AK-47 back to the middle ages.

> the models will find every possible algorithmic/design improvement

it's so hard to square such totalizing statements with my day to day experience with fable and sol, (every possible, improvement, really?? they are NOT omniscient) arguing with them/my colleagues' agents that no they have slowed down the system 200x with their terrible change, doing string operations on millions of db rows, trying to get it to understand that I don't care that it's calling it a "cache" if a cache hit is slower than what we had before.

These agents do let you learn codebases quickly, and produce code way faster. I don't look at IDEs all that often. But literally multiple times every single day I catch them doing something stupid.

I don't think its impossible that we could get better performance from the agents. I know ive tried all sorts of workflows and skills, few of which seem to have much effect on the things the models struggle with. I think a big part of it is encoding enough context for large codebases, and providing it with all the tools it needs to make it successful, things to automatically check its work, etc. But that's not automatic, in fact its generally a terrible judge of what it needs or what its bad at

Re: Managing AI Coding Costs at Scale

#122

Earlier quoted context omitted.

... where in the article did they say they were using subscriptions? I'm fairly certain enterprises can't access subscription pricing in any case, they're all API costs (Anthropic doesn't support more than 150 on subscription pricing [0][1]). [0]: https://support.claude.com/en/articles/9797531-what-is-the-e... [1]: https://support.claude.com/en/articles/9266767-what-is-the-t...

Going through their harness (codex, claude) is subscription (app use) which is heavily? subsidized. Anyone using the enterprise plan are charged the API pricing, however the article is not clear if Databricks is using enterprise plan or not which is why added the following disclaimer > Unless Databricks has some agreement in place

You can use both of those harnesses without going through subscription. That is a native feature in both Codex & Claude Code, even for non-enterprise customers.

Re: Managing AI Coding Costs at Scale

#123
post #16

I would be really curious to hear from devs at Databricks what the experience of development is like internally. I work at a small startup with essentially unlimited AI spend budget - the entire point is that I should be turning to it at every opportunity since our human labor is so expensive relative to tokens. So generally it's like: - Spend most time prioritizing/discussing what to do. - Once that's agreed, use Fa…

In our team's experience, the product of agents is generally The Homer (1). It does work, but it's vastly overengineered. When I personally want tight code, I have to spend a considerable amount of time adjusting it manually: - It needs to be trimmed down. In my experience, at least one agent I use struggles to produce minimalist designs, and it's very frustrating - I need to consider whether there are solutions base…

If software has become a mass-produced commodity then seems to me the software business will become a much more finance focused business

You will really have to weigh the cost of making the software against the expected revenue.

Re: Managing AI Coding Costs at Scale

#124
post #118

Earlier quoted context omitted.

"startup" and "legacy codebase" are diametrically opposed concepts. And if you're saying (based on your other comments) that a 6 month window is enough to create a legacy codebase...that indicates a serious lack of experience or understanding as to what a legacy codebase is, or why they exist.

Man, so many people in this thread just arguing pointless semantics, making weirdo absolutist (and incorrect) statements. Accept that other people may ascribe different meanings/interpretations to words than you, and that if your reading of their statement doesn't make sense to you, perhaps you are simply reading it wrong. Trying to hold someone else to your definition of words suits what purpose exactly? Are you jus…

I agree with your larger point. Though I think it's pretty natural to wonder how the poster ended up with a multi million line codebase.

Re: Managing AI Coding Costs at Scale

#125
post #111

Earlier quoted context omitted.

> Why aren't you guiding your LLM usage? Is that what I said - to spam it and not guide anything? Or to have a careful workflow where you agree on design and maximize your human judgement/leverage? You didn't say anything positively or negatively regarding this so I made an assumption that you were using the LLM relatively unguided (e.g. a bit of oversight, not the kind of thing that heavy code reviews used to involv…

> You didn't say anything positively or negatively regarding this so I made an assumption that you were using the LLM relatively unguided I feel like this statement betrays your lack of advanced experience coding with LLMs. OP's elaboration of the steps they are going through (planning, agreeing on plan, getting one LLM to draft execution plan, approving it, then executing with a separate LLM, then reviewing/testing)…

Planning, agreeing on a plan, separating planning and implementation LLM, using separate review LLMs, these are all table stakes. This isn't "guidance" if you're getting paid to write software. If you think "unguided" means "I typed a prompt into claude code and waited yolo" I don't know what to say but, you have a very different idea of what professionals do than I do.

I find for my own work that I need to read the diff the LLM produces then offer feedback on the diff in its own loop before I am satisfied, and this is after all the unattended QA steps through Codex Computer or Claude MCPs happen. Then auto reviewers come in and then reviewers come in. Of course, at our stage, we rarely have this luxury and it's only reserved for the very core of our codebase.

This is still much less guidance than we used to do for code before agents became popular. Even at Series A companies, before agents, we used to socialize tech specs, get buy-in from multiple engineers, create test plans, etc etc.

> Anyone making blanket statements about LLMs producing garbage is just telling on themselves about not having proper SDLC practices in place.

> I feel like this statement betrays your lack of advanced experience coding with LLMs.

Are we in school debate club? I don't know what's going on lol, I'm just curious how people are using LLMs! Is it just that irresistable to take a cheap shot at each other?

Re: Managing AI Coding Costs at Scale

#126
post #50
post #37

Earlier quoted context omitted.

This was more true a few months ago but Fable has improved the situation considerably. Also just remember - minimalist code looks and feels great but customers do not read your code. I have caught myself many times providing "corrections" to abstractions that were already ~fine, just not perfect. The average SWE costs $200/hr. Careful you don't burn $50 worrying about code that will likely be rewritten or can be bett…

> The average SWE costs $200/hr. And this is how I find out I'm woefully underpaid.

That "cost" includes all the overhead provided by the company: benefits, rent for offices, utilities, equipment, etc. The average SWE is not taking home anything close to that, outside of Silicon Valley and a few other limited areas.

Re: Managing AI Coding Costs at Scale

#127
post #16

I would be really curious to hear from devs at Databricks what the experience of development is like internally. I work at a small startup with essentially unlimited AI spend budget - the entire point is that I should be turning to it at every opportunity since our human labor is so expensive relative to tokens. So generally it's like: - Spend most time prioritizing/discussing what to do. - Once that's agreed, use Fa…

In our team's experience, the product of agents is generally The Homer (1). It does work, but it's vastly overengineered. When I personally want tight code, I have to spend a considerable amount of time adjusting it manually: - It needs to be trimmed down. In my experience, at least one agent I use struggles to produce minimalist designs, and it's very frustrating - I need to consider whether there are solutions base…

> software production has become a mass-produced commodity

For who?

The public? The public has never liked buying software at any price.

Businesses? Businesses need higher quality software when it's relevant to their core competencies, so they hire people instead. Buying competing SaaS or depending too much on AI is throwing the baby out with the bathwater.

Re: Managing AI Coding Costs at Scale

#128
post #36
post #32

Earlier quoted context omitted.

Disagree. I operate this way inside a multi-million line legacy codebase.

> I work at a small startup How does a “small startup” end up with a multi million line “legacy” codebase? Something not mathing

You'd be surprised. I met a guy last week who was proud to tell me he had vibe coded an almost 2 million line code base. The app did not sound that complicated, so I'm assuming it's full of copy-pasta flavored slop.

Re: Managing AI Coding Costs at Scale

#129
post #62

Earlier quoted context omitted.

No, actually. The point is to build a profitable business.

How long has your startup been around? I’ve worked at plenty of startups over the past 20 years. Including one that was still calling themselves a startup 10 years out. The org I work at now was a startup before my tech giant employer acquired them. We have a very bloated and very profitable 8 year old codebase that is barely 500k LOC. I’ve never seen a startup with a multi million line legacy codebase.

They may have forked something

Re: Managing AI Coding Costs at Scale

#130
post #49

Earlier quoted context omitted.

> The average SWE costs $200/hr This is a pointless quibble but the hourly rate claim is not true--it's like ~$60 in the USA [0]. Maybe you meant at a specific Org but this is important context when comparing "pricing" between human and AI. [0] https://www.salaryexpert.com/salary/job/software-developer/u...

How is this not true? Taking a Senior SWE @ ~$200K, even just the base salary cost / 2080 working hours is $100/hr. Fully loaded employer cost + accounting for non-coding time gets you to upper 100s easily. Even for a junior making $100K, I have a hard time believe their time is worth less than $75/hr or so. Edit: Fine, "Senior" is not "Average". But naive salary is not the true numerator.

Of course, the SWEs making that much (over 200k) are not representative of the broader field. That's the point.

Pay hits a ceiling, and that ceiling is moving lower regardless of experience. That has nothing to do with AI, but what the market will bear. Hiring counts of humans must increase no matter what. Moving some of the spend to AI reduces the risk of hiring less qualified employees they might have rejected a decade ago.

Wages at the top end are stagnating to subsidize this. That's undeniable.

Post reply on HN