Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Managing AI Coding Costs at Scale

databricks.com

Post reply on HN
Print view View original

21–30 of 264 posts

Re: Managing AI Coding Costs at Scale

#21

Post by reqo »

extr wrote: post #16

I would be really curious to hear from devs at Databricks what the experience of development is like internally. I work at a small startup with essentially unlimited AI spend budget - the entire point is that I should be turning to it at every opportunity since our human labor is so expensive relative to tokens. So generally it's like: - Spend most time prioritizing/discussing what to do. - Once that's agreed, use Fa…

IME this works until it does not. This approach works well at the beginning of a greenfield project, but at the same time because it is so easy to add features, you will likely ship something that is way too over engineered. And that complexity will not amortize over next increments and will more likely lead to the entire project being a black box only fully understood by AI. However a more careful use of AI for targeted surgical changes is far more ”productive” in the long term IMO.

Re: Managing AI Coding Costs at Scale

#22

Post by catlover76 »

extr wrote: post #16

I would be really curious to hear from devs at Databricks what the experience of development is like internally. I work at a small startup with essentially unlimited AI spend budget - the entire point is that I should be turning to it at every opportunity since our human labor is so expensive relative to tokens. So generally it's like: - Spend most time prioritizing/discussing what to do. - Once that's agreed, use Fa…

[dead]

Re: Managing AI Coding Costs at Scale

#23

Post by ChoosesBarbecue »

sandeepkd wrote: post #14

I find this funny and interesting at some levels 1. Codex, Claude and others try to switch models being used at their level itself to manage the cost and outcomes 2. Now company like data bricks develops one more layer on the top of it to do the same task, of finding the base harness and applicable model Companies like Codex and Claude are focussing/investing heavily on to ensure that people are using their harness d…

... where in the article did they say they were using subscriptions? I'm fairly certain enterprises can't access subscription pricing in any case, they're all API costs (Anthropic doesn't support more than 150 on subscription pricing [0][1]).

[0]: https://support.claude.com/en/articles/9797531-what-is-the-e...

[1]: https://support.claude.com/en/articles/9266767-what-is-the-t...

Re: Managing AI Coding Costs at Scale

#25

Post by justincormack »

sandeepkd wrote: post #14

I find this funny and interesting at some levels 1. Codex, Claude and others try to switch models being used at their level itself to manage the cost and outcomes 2. Now company like data bricks develops one more layer on the top of it to do the same task, of finding the base harness and applicable model Companies like Codex and Claude are focussing/investing heavily on to ensure that people are using their harness d…

Databricks will be using the API anyway, thats all you get with an enterprise agreement.

Re: Managing AI Coding Costs at Scale

#26

Post by extr »

RugnirViking wrote: post #20
extr wrote: post #16

I would be really curious to hear from devs at Databricks what the experience of development is like internally. I work at a small startup with essentially unlimited AI spend budget - the entire point is that I should be turning to it at every opportunity since our human labor is so expensive relative to tokens. So generally it's like: - Spend most time prioritizing/discussing what to do. - Once that's agreed, use Fa…

Do you have issues with performance at the moment? Right now I tend to find that it produces absolutely terrible design patterns and especially performance. I mean maybe I don't know exactly what area you're looking at but yeah for us we tend to find it's terrible wrt dB/caching/scaling and often any performance improvements it proposes end up actually shooting itself in the foot and being worse than before but it's…

Performance is better than ever. It's never been more practical to set up wildly complex synthetic test environments and measure perf wins. Plus the models will find every possible algorithmic/design improvement.

It actually gives me quite an uncanny feeling, bulldozing over years of human optimization work with a newer, "perfect" design. Like bringing an AK-47 back to the middle ages.

Re: Managing AI Coding Costs at Scale

#27

Post by pizza234 »

extr wrote: post #16

I would be really curious to hear from devs at Databricks what the experience of development is like internally. I work at a small startup with essentially unlimited AI spend budget - the entire point is that I should be turning to it at every opportunity since our human labor is so expensive relative to tokens. So generally it's like: - Spend most time prioritizing/discussing what to do. - Once that's agreed, use Fa…

In our team's experience, the product of agents is generally The Homer (1). It does work, but it's vastly overengineered.

When I personally want tight code, I have to spend a considerable amount of time adjusting it manually:

- It needs to be trimmed down. In my experience, at least one agent I use struggles to produce minimalist designs, and it's very frustrating

- I need to consider whether there are solutions based on higher-level assumptions, that AIs typically miss

- I need to check whether there are off-the-shelf solutions - AIs like to reinvent the wheel

IMO, software production has become a mass-produced commodity in every sense - it's much more expensive to produce software manually, but the quality is not the same.

(1) https://simpsons.fandom.com/wiki/The_Homer

Re: Managing AI Coding Costs at Scale

#28

Post by the_sleaze_ »

biophysboy wrote: post #18
extr wrote: post #16

I would be really curious to hear from devs at Databricks what the experience of development is like internally. I work at a small startup with essentially unlimited AI spend budget - the entire point is that I should be turning to it at every opportunity since our human labor is so expensive relative to tokens. So generally it's like: - Spend most time prioritizing/discussing what to do. - Once that's agreed, use Fa…

Do you have tips for generating clean productive output per dollar?

in my humble experience it boils down to mastery.

Are you at least conversational in the subject matter? You're gonna have a good time just by paying attention and adjusting your workflow. If you're getting a lot of back and forth with it, its asking a lot of planning type questions, stop, step back, rethink the whole feature, and start again from the beginning with everything more fleshed out.

If you are in a brand new field, there's no way to bridge that divide. The issue is you don't know what is good or bad, or whether what you have learned is good or bad. You're in a sports car and you don't know how to drive much less what's track and what's field.

You can spend a lot of effort getting good at prompting towards writing tests and E2E tests to at least verify your app does what you expect it to, regardless of experience.

Re: Managing AI Coding Costs at Scale

#29

Post by notduckrabbit »

aliasxneo wrote: post #7

First time hearing of Omnigent. Anyone have experience using it?

I've tested Omnigent superficially, attracted to its thinking around policy, governance, sandboxing, and ui. But it's still alpha at present. I forked its Polly model and got working a somewhat more complex multiagent workflow that I've also modeled in Sandcastle and Gas City but the agent broke after the next update which I would have needed to patch to maintain functionality. Subjectively I also noticed individual models seemed to be performing somewhat worse when wrapped in the platform's framework, presumably due to the extra context introduced (token use was measurably higher). Promising project that I'll revisit when it's further along and I do not doubt the outcomes Databricks claims in committedly dogfooding it.

Re: Managing AI Coding Costs at Scale

#30

Post by jchook »

extr wrote: post #16

I would be really curious to hear from devs at Databricks what the experience of development is like internally. I work at a small startup with essentially unlimited AI spend budget - the entire point is that I should be turning to it at every opportunity since our human labor is so expensive relative to tokens. So generally it's like: - Spend most time prioritizing/discussing what to do. - Once that's agreed, use Fa…

This is very close to my workflow but you forgot one important step:

- Suggest a better approach that makes the AI say, “That’s much simpler. And you’re right. My original plan was over-engineered.”

Post reply on HN