Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Managing AI Coding Costs at Scale

databricks.com

Post reply on HN
Print view View original

151–160 of 269 posts

Re: Managing AI Coding Costs at Scale

#151

Post by extr »

RugnirViking wrote: post #121
extr wrote: post #26

Earlier quoted context omitted.

Performance is better than ever. It's never been more practical to set up wildly complex synthetic test environments and measure perf wins. Plus the models will find every possible algorithmic/design improvement. It actually gives me quite an uncanny feeling, bulldozing over years of human optimization work with a newer, "perfect" design. Like bringing an AK-47 back to the middle ages.

> the models will find every possible algorithmic/design improvement it's so hard to square such totalizing statements with my day to day experience with fable and sol, (every possible , improvement, really?? they are NOT omniscient) arguing with them/my colleagues' agents that no they have slowed down the system 200x with their terrible change, doing string operations on millions of db rows, trying to get it to unde…

yeah it's true, you do have to guide them. i find that the key is you have to know what's possible. you have to have the instinct for "this really shouldn't be so difficult". my junior SWE coworkers have the same trouble as your coworkers.

but the revolution is it doesn't take that long. in like 15 minutes you can chat with fable and get to the meat of whatever the issue is with repeated questioning. and then it does the solution for you. so it's not magic but it's still like a 100x speedup.

Re: Managing AI Coding Costs at Scale

#152

Post by sbarre »

Karrot_Kream wrote: post #125
sbarre wrote: post #111

Earlier quoted context omitted.

> You didn't say anything positively or negatively regarding this so I made an assumption that you were using the LLM relatively unguided I feel like this statement betrays your lack of advanced experience coding with LLMs. OP's elaboration of the steps they are going through (planning, agreeing on plan, getting one LLM to draft execution plan, approving it, then executing with a separate LLM, then reviewing/testing)…

Planning, agreeing on a plan, separating planning and implementation LLM, using separate review LLMs, these are all table stakes. This isn't "guidance" if you're getting paid to write software. If you think "unguided" means "I typed a prompt into claude code and waited yolo" I don't know what to say but, you have a very different idea of what professionals do than I do. I find for my own work that I need to read the…

> Are we in school debate club?

Not that I know of but that's the conclusion I drew from your statement.

It's not a cheap shot unless you took it personally?

I suppose I could have said "the fact that OP's explanation of how they work did not lead you to conclude they were in fact guiding their LLM usage quite a bit tells me that perhaps you have not been working with LLMs in any advanced capacity".

For the SDLC comment I admit it was a broader statement (based on observing people generalizing that "LLMs produce bad outputs") and not specifically aimed at you, and I didn't make that clear, so my bad.

Re: Managing AI Coding Costs at Scale

#153

Post by oblio »

roncesvalles wrote: post #148
pianopatrick wrote: post #123

Earlier quoted context omitted.

If software has become a mass-produced commodity then seems to me the software business will become a much more finance focused business You will really have to weigh the cost of making the software against the expected revenue.

Since software is still a winner-takes-all market, the mass-production property of software doesn't really matter. In such markets, what you produce is either worth nothing or worth millions of dollars. For as long as it's the case that well-constructed code (with or without LLM help) is more likely to be in the latter category, the economics of software don't really change. Even before LLMs, you could've commissione…

> Since software is still a winner-takes-all market, the mass-production property of software doesn't really matter.

Fairly sure software dev was always an iceberg. Most software and most software devs aren't working on horizontal software, but on vertical software, in cost centers. Sales for that kind of software don't scale as much.

Re: Managing AI Coding Costs at Scale

#154

Post by DenisM »

notduckrabbit wrote: post #29
aliasxneo wrote: post #7

First time hearing of Omnigent. Anyone have experience using it?

I've tested Omnigent superficially, attracted to its thinking around policy, governance, sandboxing, and ui. But it's still alpha at present. I forked its Polly model and got working a somewhat more complex multiagent workflow that I've also modeled in Sandcastle and Gas City but the agent broke after the next update which I would have needed to patch to maintain functionality. Subjectively I also noticed individual…

Double-harnessing distortions seems like a weak point. I wonder if it’s just a temporary measure and long term it’s about writing custom harness going straight to inference APIs across all models.

Re: Managing AI Coding Costs at Scale

#155

Post by roncesvalles »

oblio wrote: post #153
roncesvalles wrote: post #148

Earlier quoted context omitted.

Since software is still a winner-takes-all market, the mass-production property of software doesn't really matter. In such markets, what you produce is either worth nothing or worth millions of dollars. For as long as it's the case that well-constructed code (with or without LLM help) is more likely to be in the latter category, the economics of software don't really change. Even before LLMs, you could've commissione…

> Since software is still a winner-takes-all market, the mass-production property of software doesn't really matter. Fairly sure software dev was always an iceberg. Most software and most software devs aren't working on horizontal software, but on vertical software, in cost centers. Sales for that kind of software don't scale as much.

Most vertical software is just horizontal software glued together

Re: Managing AI Coding Costs at Scale

#156

Post by fragmede »

copperx wrote: post #47
dieselgate wrote: post #45

Earlier quoted context omitted.

> The average SWE costs $200/hr This is a pointless quibble but the hourly rate claim is not true--it's like ~$60 in the USA [0]. Maybe you meant at a specific Org but this is important context when comparing "pricing" between human and AI. [0] https://www.salaryexpert.com/salary/job/software-developer/u...

$15, where we're going.

Federal minimum wage is $7.25 per hour

Re: Managing AI Coding Costs at Scale

#157

Post by deathmonger5000 »

aliasxneo wrote: post #7

First time hearing of Omnigent. Anyone have experience using it?

If you’re into web based meta-harnesses you might like Circus Chief: https://github.com/ferrislucas/Circus-Chief

Similar idea re using the same UI across all models, but the agent can modify the harness config as well as start/schedule sessions, etc. The Kanban board feature can be used to orchestrate agent driven workflows, and the agent itself can modify the Kanban lanes. Basically the agent can do all the same things you use the UI for.

Example prompt: “schedule a session using Opus 5 with max thinking for every feature in the PRD on the canvas. For every session make the prompt instruct the agent to review end to end test coverage for the feature and create a report on the canvas for test gaps that you find. Schedule the sessions overnight tonight spread evenly throughout the night, and have each session set to retry upon token exhaustion.”

I think it’s more meta than other meta-harnesses, but I’m biased because it’s my pet project.

Re: Managing AI Coding Costs at Scale

#158

Post by bdangubic »

extr wrote: post #16

I would be really curious to hear from devs at Databricks what the experience of development is like internally. I work at a small startup with essentially unlimited AI spend budget - the entire point is that I should be turning to it at every opportunity since our human labor is so expensive relative to tokens. So generally it's like: - Spend most time prioritizing/discussing what to do. - Once that's agreed, use Fa…

do the same across 20 terminals (as you should) and now you are up to $1.6k/day. would that give you output or 60-80 engineers? not a chance, right?

no one’s AI spent will be in question working a single terminal with carefully planned out and executed process you do

Re: Managing AI Coding Costs at Scale

#159

Post by fragmede »

slopinthebag wrote: post #104
extr wrote: post #37

Earlier quoted context omitted.

This was more true a few months ago but Fable has improved the situation considerably. Also just remember - minimalist code looks and feels great but customers do not read your code. I have caught myself many times providing "corrections" to abstractions that were already ~fine, just not perfect. The average SWE costs $200/hr. Careful you don't burn $50 worrying about code that will likely be rewritten or can be bett…

The average SWE makes $400k a year? Are you being serious?

Costs, not makes. That includes employer taxes, benefits, offices, travel, training, hardware, any perks.

Re: Managing AI Coding Costs at Scale

#160

Post by mortalapeman »

salmonfamine wrote: post #102

I think there is a lot of dev cope in this thread. My workflow is very simple: 1. develop requirements for code change 2. take manual notes for implementation, maybe use LLM for some discovery/investigation 3. present notes to frontier LLM 4. develop implementation plan (bulk of work) 5. let LLM rip 6. review diff, manually fixing/refactoring code as necessary, sometimes prompting for revisions 7. get automated LLM r…

This has been my experience as well. The best model I have access to right now is Opus 4.8. It's really good at fixing bugs in an established architecture or adding a similar feature, but it's absolutely mid tier at putting together a simple effective architecture for even common software problems. I can tell that functional programming practices are not a large part of it's training material.
Post reply on HN