Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Managing AI Coding Costs at Scale

databricks.com

Post reply on HN
Print view View original

41–50 of 268 posts

Re: Managing AI Coding Costs at Scale

#41

Post by nujabe »

extr wrote: post #16

I would be really curious to hear from devs at Databricks what the experience of development is like internally. I work at a small startup with essentially unlimited AI spend budget - the entire point is that I should be turning to it at every opportunity since our human labor is so expensive relative to tokens. So generally it's like: - Spend most time prioritizing/discussing what to do. - Once that's agreed, use Fa…

> essentially unlimited AI spend budget

> I probably spend like $80 a day

This doesn’t sound like “unlimited”, I spend more than this out of pocket per day and I have a strict budget.

Re: Managing AI Coding Costs at Scale

#42

Post by seizethecheese »

platinumrad wrote: post #3

Careful. If you admit to using models that weren't trained by OpenAI or Anthropic then you might hauled in front of Congress: https://www.scmp.com/news/china/diplomacy/article/3362616/us...

I'd prefer congress to be asking questions (this is all they are doing so far, based on the article) before doing any legislating.

Re: Managing AI Coding Costs at Scale

#43

Post by dieselgate »

pizza234 wrote: post #27
extr wrote: post #16

I would be really curious to hear from devs at Databricks what the experience of development is like internally. I work at a small startup with essentially unlimited AI spend budget - the entire point is that I should be turning to it at every opportunity since our human labor is so expensive relative to tokens. So generally it's like: - Spend most time prioritizing/discussing what to do. - Once that's agreed, use Fa…

In our team's experience, the product of agents is generally The Homer (1). It does work, but it's vastly overengineered. When I personally want tight code, I have to spend a considerable amount of time adjusting it manually: - It needs to be trimmed down. In my experience, at least one agent I use struggles to produce minimalist designs, and it's very frustrating - I need to consider whether there are solutions base…

It took me a few seconds of deliberating if The Homer was a reference to baseball or "The Odyssey" and then realized there was a footnote

> software production has become a mass-produced commodity in every sense - it's much more expensive to produce software manually, but the quality is not the same.

Agreed.

Re: Managing AI Coding Costs at Scale

#44

Post by nujabe »

extr wrote: post #38
nujabe wrote: post #36

Earlier quoted context omitted.

> I work at a small startup How does a “small startup” end up with a multi million line “legacy” codebase? Something not mathing

Have you worked at many startups?

No, but not relevant.

What is the point of working at a startup if you’re dealing with millions of lines of legacy code ? Isn’t the whole point of startups to create & innovate with a clean slate and modern tools?

Re: Managing AI Coding Costs at Scale

#45

Post by dieselgate »

extr wrote: post #37
pizza234 wrote: post #27

Earlier quoted context omitted.

In our team's experience, the product of agents is generally The Homer (1). It does work, but it's vastly overengineered. When I personally want tight code, I have to spend a considerable amount of time adjusting it manually: - It needs to be trimmed down. In my experience, at least one agent I use struggles to produce minimalist designs, and it's very frustrating - I need to consider whether there are solutions base…

This was more true a few months ago but Fable has improved the situation considerably. Also just remember - minimalist code looks and feels great but customers do not read your code. I have caught myself many times providing "corrections" to abstractions that were already ~fine, just not perfect. The average SWE costs $200/hr. Careful you don't burn $50 worrying about code that will likely be rewritten or can be bett…

> The average SWE costs $200/hr

This is a pointless quibble but the hourly rate claim is not true--it's like ~$60 in the USA [0]. Maybe you meant at a specific Org but this is important context when comparing "pricing" between human and AI.

[0] https://www.salaryexpert.com/salary/job/software-developer/u...

Re: Managing AI Coding Costs at Scale

#46

Post by manmal »

RugnirViking wrote: post #20
extr wrote: post #16

I would be really curious to hear from devs at Databricks what the experience of development is like internally. I work at a small startup with essentially unlimited AI spend budget - the entire point is that I should be turning to it at every opportunity since our human labor is so expensive relative to tokens. So generally it's like: - Spend most time prioritizing/discussing what to do. - Once that's agreed, use Fa…

Do you have issues with performance at the moment? Right now I tend to find that it produces absolutely terrible design patterns and especially performance. I mean maybe I don't know exactly what area you're looking at but yeah for us we tend to find it's terrible wrt dB/caching/scaling and often any performance improvements it proposes end up actually shooting itself in the foot and being worse than before but it's…

Are you using the SOTA models at very high reasoning during planning? IME that makes a LOT of a difference. I‘d also never let them just rip into the architecture, but always push back and ask for alternatives first. Once the overall plan is nailed, not that much can go wrong. Provided it’s a reasonable change set and not a 20k LOC PR.

Re: Managing AI Coding Costs at Scale

#47

Post by copperx »

dieselgate wrote: post #45
extr wrote: post #37

Earlier quoted context omitted.

This was more true a few months ago but Fable has improved the situation considerably. Also just remember - minimalist code looks and feels great but customers do not read your code. I have caught myself many times providing "corrections" to abstractions that were already ~fine, just not perfect. The average SWE costs $200/hr. Careful you don't burn $50 worrying about code that will likely be rewritten or can be bett…

> The average SWE costs $200/hr This is a pointless quibble but the hourly rate claim is not true--it's like ~$60 in the USA [0]. Maybe you meant at a specific Org but this is important context when comparing "pricing" between human and AI. [0] https://www.salaryexpert.com/salary/job/software-developer/u...

$15, where we're going.

Re: Managing AI Coding Costs at Scale

#48

Post by Karrot_Kream »

extr wrote: post #38
nujabe wrote: post #36

Earlier quoted context omitted.

> I work at a small startup How does a “small startup” end up with a multi million line “legacy” codebase? Something not mathing

Have you worked at many startups?

Something isn't clear about the size of your codebase here and the level of reliability your customers expect, as a reader of your comments. Clarity there will help.

My observation has been:

- Initial greenfield work by an LLM is fast and very effective with minimal or no human oversight.

- Subsequent work ends up being over engineered and very verbose. Assumptions are made that aren't suited to the problem at hand (for example I find Fable is extremely regex happy where structured data would work much better from a readability perspective.)

- Once code bloats beyond a certain point due to unguided LLM usage, complexity is high enough that only LLMs can operate on the codebase with any economical amount of time.

- Rinse repeat and your code ends up unclear about any state that's not explicitly being tested and verified in QA loops

For some of our products this has been fine, for others it's been problematic. An understanding of your size and reliability requirements will help make the conversation more productive.

Re: Managing AI Coding Costs at Scale

#49

Post by extr »

dieselgate wrote: post #45
extr wrote: post #37

Earlier quoted context omitted.

This was more true a few months ago but Fable has improved the situation considerably. Also just remember - minimalist code looks and feels great but customers do not read your code. I have caught myself many times providing "corrections" to abstractions that were already ~fine, just not perfect. The average SWE costs $200/hr. Careful you don't burn $50 worrying about code that will likely be rewritten or can be bett…

> The average SWE costs $200/hr This is a pointless quibble but the hourly rate claim is not true--it's like ~$60 in the USA [0]. Maybe you meant at a specific Org but this is important context when comparing "pricing" between human and AI. [0] https://www.salaryexpert.com/salary/job/software-developer/u...

How is this not true? Taking a Senior SWE @ ~$200K, even just the base salary cost / 2080 working hours is $100/hr. Fully loaded employer cost + accounting for non-coding time gets you to upper 100s easily.

Even for a junior making $100K, I have a hard time believe their time is worth less than $75/hr or so.

Edit: Fine, "Senior" is not "Average". But naive salary is not the true numerator.

Re: Managing AI Coding Costs at Scale

#50

Post by j-bos »

extr wrote: post #37
pizza234 wrote: post #27

Earlier quoted context omitted.

In our team's experience, the product of agents is generally The Homer (1). It does work, but it's vastly overengineered. When I personally want tight code, I have to spend a considerable amount of time adjusting it manually: - It needs to be trimmed down. In my experience, at least one agent I use struggles to produce minimalist designs, and it's very frustrating - I need to consider whether there are solutions base…

This was more true a few months ago but Fable has improved the situation considerably. Also just remember - minimalist code looks and feels great but customers do not read your code. I have caught myself many times providing "corrections" to abstractions that were already ~fine, just not perfect. The average SWE costs $200/hr. Careful you don't burn $50 worrying about code that will likely be rewritten or can be bett…

> The average SWE costs $200/hr.

And this is how I find out I'm woefully underpaid.

Post reply on HN