good engineer + llm = good engineer.
bad engineer + llm = bad engineer.
Managing AI Coding Costs at Scale
171–180 of 269 posts
Re: Managing AI Coding Costs at Scale
#172I would be really curious to hear from devs at Databricks what the experience of development is like internally. I work at a small startup with essentially unlimited AI spend budget - the entire point is that I should be turning to it at every opportunity since our human labor is so expensive relative to tokens. So generally it's like: - Spend most time prioritizing/discussing what to do. - Once that's agreed, use Fa…
I was just about to say, how could routing possibly be worth it at the risk that the work output is sub par?
Re: Managing AI Coding Costs at Scale
#173I suspect that when it comes to hard complex software products, you’re better off ignoring agents and doing “trad coding”. What you lose in short term speed you gain in manageable complex codebases.
If you have a 500k line codebase and even > 50% is written by agents, you are in a world of pain that won’t justify the costs longer term.
Now of course, there are products that just involve lots of code but are not actually complex. This is generally the project with like hundreds or thousands of features but most of the features are separate and don’t actually interact in complex ways. Think a task management app with hundreds of features like calendar, email integration etc. there I think agents gives you more bang for the buck. Just my thought, using agents at work.
Re: Managing AI Coding Costs at Scale
#174Earlier quoted context omitted.
Since software is still a winner-takes-all market, the mass-production property of software doesn't really matter. In such markets, what you produce is either worth nothing or worth millions of dollars. For as long as it's the case that well-constructed code (with or without LLM help) is more likely to be in the latter category, the economics of software don't really change. Even before LLMs, you could've commissione…
I think part of the reason software was winner take most was the difficulty of making software. I remember hearing a story that in the past movies were so technically difficult to make that any movie that got made had a good chance to be a profitable hit. But as movies got cheaper to make, more movies got made. Nowadays movie studio execs have to really calculate out the audience and expected revenue for any new movi…
That honestly sounds a lot healthier than “just ship what the CTO/Product team wants” with as much hand waving as is necessary to very roughly estimate ROI and then pray it hits with the market. In anything that’s not a startup operating in a new industry, the “old way” is a hard way to run a business
Re: Managing AI Coding Costs at Scale
#175I suspect that when it comes to hard complex software products, you’re better off ignoring agents and doing “trad coding”. What you lose in short term speed you gain in manageable complex codebases. If you have a 500k line codebase and even > 50% is written by agents, you are in a world of pain that won’t justify the costs longer term. Now of course, there are products that just involve lots of code but are not actua…
> if you have a 500k line codebase and even > 50% is written by agents, you are in a world of pain that won’t justify the costs longer term
A bold claim to make with little to no supporting evidence
Re: Managing AI Coding Costs at Scale
#176not sure about the use of exponential and efficiency frontier here, these have formal sides to them but seem to be used rather inflationary and colloquially.
Re: Managing AI Coding Costs at Scale
#177Earlier quoted context omitted.
In our team's experience, the product of agents is generally The Homer (1). It does work, but it's vastly overengineered. When I personally want tight code, I have to spend a considerable amount of time adjusting it manually: - It needs to be trimmed down. In my experience, at least one agent I use struggles to produce minimalist designs, and it's very frustrating - I need to consider whether there are solutions base…
No surprise, LLM companies optimize for waste. More tokens, and more prompts means more revenue. Reminds of Google’s Prabhakar Raghavan story: deliberately making search worse [1] [1]: https://pluralistic.net/2024/04/24/naming-names/#prabhakar-r...
Or, more likely, it's that concise code requires a much deeper, wholistic, understanding that these models just are capable of yet.
Same with a junior dev. They don't write long form spaghetti because they're trying to write more LOC. They do it because not doing it is hard, literally above their pay grade.
I use LLM every day, but they're still completely awful at architecture. I don't think this clear lack of ability is some conspiracy.
Re: Managing AI Coding Costs at Scale
#178Reads like an add to Omnigent or whatever harness (wait it’s meta harness?.
Re: Managing AI Coding Costs at Scale
#179Earlier quoted context omitted.
> You didn't say anything positively or negatively regarding this so I made an assumption that you were using the LLM relatively unguided I feel like this statement betrays your lack of advanced experience coding with LLMs. OP's elaboration of the steps they are going through (planning, agreeing on plan, getting one LLM to draft execution plan, approving it, then executing with a separate LLM, then reviewing/testing)…
Planning, agreeing on a plan, separating planning and implementation LLM, using separate review LLMs, these are all table stakes. This isn't "guidance" if you're getting paid to write software. If you think "unguided" means "I typed a prompt into claude code and waited yolo" I don't know what to say but, you have a very different idea of what professionals do than I do. I find for my own work that I need to read the…
> If you think "unguided" means "I typed a prompt into claude code and waited yolo" I don't know what to say but, you have a very different idea of what professionals do than I do.
What exactly does "unguided" mean to you, then?
Re: Managing AI Coding Costs at Scale
#180Earlier quoted context omitted.
Open ai is allowing subscription use, anthropic also paused the effort to stop subscription use.
They did? Is there a source where I can learn more? I'd love to use my Anthropic subscription with opencode.
Only if opencode uses the agent sdk/Claude -p
June 15 changes would be the keyword to check, but at least in anthropics case it's... Complicated