Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Using AI to write better code more slowly

nolanlawson.com

Post reply on HN
Print view View original

481–490 of 511 posts

Re: Using AI to write better code more slowly

#481

Post by RevEng »

suprjami wrote: post #472
hrideshmg wrote: post #381

As a junior, i do actually enjoy going back and forth with the AI discussing different ways to implement something and exploring alternatives. More often than not, I'd have an architectural idea that I'm not that confident in. The process of talking with the LLM takes a long time but it helps me sharpen the initial approach or even come up with a new one depending on the requirements.

When doing this, I particularly like that the LLM sometimes gets things wrong. It forces me to really understand each thing deeply so that I evaluate it properly. It is like taking an exam where the exam writer is hostile and sneaks in trick questions. You only spot that the question is wrong when you fully reason through the answer.

Huh, good point! When a colleague asks me to review their design or otherwise discuss it, I'm always looking for things they might have missed, assumptions they silently made, or corner cases that could come up. I start from the position that there is likely something missing and I need to find out what. Likewise, when I'm looking at suggestions or code or anything else from an AI, I'm assuming it made some mistakes, made some unstated assumptions, or didn't consider some corner cases, and so I'm having to carefully think through what it says to spot the mistake, rather than casually skimming it and going, "LGTM!" If it were too reliable, I might get lazy and not look too hard knowing that it's probably right anyway so there's no point trying too hard to find something. It's the same thing my juniors will sometimes do to me: don't assume I'm right just because I'm experienced - I still make mistakes too! I want to be questioned on anything that might not make sense, because even if it was intentional, the fact that the reason isn't clear is itself a problem to resolve. And I only know so many things - we all have different experiences and a junior can have just as much they can teach me as a senior.

Re: Using AI to write better code more slowly

#482

Post by RevEng »

bonoboTP wrote: post #293
jillesvangurp wrote: post #249

Regardless of what model you use, agentic coding tools are indeed pretty good at finding issues if you target them a bit. And they have no respect for their own code or any sense of shame. So, you can just point them at their own code with a new thread. Many AI models seem biased to cutting corners by default when generating code, even when you ask them not to. But a few simple follow up prompts can address that. Sim…

My problem is that it "finds issues" all the time and it never really ends. You go through the list, make a decision on how to go about it, give it back to the AI, it does the changes, you ask for issues again, there are now new issues in part due to the solutions from the previous fixes, now you again assess each issue and it's often valid but you have to ask yourself if it's worth fixing right now and whether the f…

I treat it like other triage tasks: things could always be better, but how much effort does it take and how much better could it be?

There's a common saying that the enemy of good is perfect. It's easy to get stuck in the loop of endlessly polishing something but never actually releasing it, even without AI. It's on us to decide how good is good enough and when to stop.

Over time I've learned to be rather aggressive about cutting out work. I'll quickly ask myself how serious is the issue (does it give wrong answers? block important flows? look embarrassing? or is it just a minor annoyance?) and how much effort would it take (five minutes? two hours? three weeks?). I should be able to make that call in no more than 30 seconds. I skim through the list of 20 suggestions the AI gives, I make plans to iterate on the 3 that are serious, and I simply accept that the rest are "good enough". It's not easy - both to be willing to let issues stand and to make the decision about what is good enough - but it's an important part of the job when triaging lists of bug reports and feature requests, so it's something we need to get good at anyway.

Re: Using AI to write better code more slowly

#483

Post by RevEng »

themanmaran wrote: post #80

As I read this, I'm also working through a pretty dense feature that took a fair bit of iteration. The end result is actually significantly less code than it was about halfway through. And I was wondering if the AI actually helped me at all, since surely I could have written the code in the same time it took to iterate But! Because of AI I was able to rapidly hack out like 4 variants of this feature that I didn't lik…

Great point! This is along the same lines as a low fidelity prototype. It doesn't have to be production quality - hell, it barely needs to work so long as it's good enough to get feedback. Now I can have higher fidelity prototypes in the same time or more iterations in the same time, either of which tend to give me more insight and get me closer to the solution faster. Even if I never ship a line of AI-generated code, I can use it to write the same throw away code I did before, but much faster.

Re: Using AI to write better code more slowly

#484

Post by RevEng »

hintymad wrote: post #115

On the other hand, some companies are pushing the idea that engineers should build robust self-evaluating agent pipeline with human feedback in the loop so that agents write most of the production code. Creao's CEO said that they rearchitected their entire production systems in two weeks this January. He also claimed that their agents implemented so many features so fast that they had to wait their business developme…

I'm always hesitant of these claims. Sure, it's possible that AI really did help them achieve the same level of quality at 100x the pace. It's also possible it generated a huge tech debt that only passes the tests but hasn't planned for future maintainability, readability, and extensibility, and a year from now their entire process will grind to a halt.

I have a few people on my team who move 5-10x faster than others in writing code. They also generate 5-10x as many bugs and require that much more rework in the things that were shipped. They move fast and break things. Their code is almost malicious compliance in that it passes the tests or spec as given, while leaving glaring holes in things that weren't fully specified. A more careful developer would have asked questions, considered alternatives, and looked for ways to leverage existing solutions or plan for future work, but that takes time now and its benefits don't show up until later.

So while I don't immediately disbelieve that 10x+ speedups are possible with heavily AI-augmented flows, I am skeptical of any short term success stories until we have time to see the long term effects. We already know that cutting corners can save time in the short term only to cost us several times more in the long term.

Re: Using AI to write better code more slowly

#485

Post by RevEng »

sailfast wrote: post #429

This approach works well, and leads to better code being shipped. The key disconnect for me is not always the code being of high quality, but ensuring my understanding of the "Why" of the bug and the fix is good enough to justify it and then also explain it if the time comes. That said, I'm learning to let go as much as I can and trust these things when it's "safe" and seeing how that shakes out. The risk is somethin…

I think we can do both. Ask the AI to summarize it, but to show examples and point out where things happen. Let it make you a Coles Notes. You still need to look at the code yourself and understand it, but an initial outline and explanation can really jump start the process and save a lot of time. Likewise, it's hard to find a bug and come up with a fix, but those same things are often oblivious in hindsight. Once we have an idea of what the bug is, we can often look at the code directly or write tests to confirm in a fraction of the time it took to discover in the first place. Once we have an idea for a fix with code and an explanation for how it fixes the bug, we can often review that explanation, think through any implications, and test the fix in a fraction of the time it takes to come up with it in the first place.

I'm happy to let the AI explore possibilities for me, eliminating the search problem. It's still on me to understand the solution, verify it works, and handle any other considerations I know of that the AI wouldn't. It gives me the insight, but I'm responsible for the final solution.

Re: Using AI to write better code more slowly

#486

Post by RevEng »

mocamoca wrote: post #334
kordlessagain wrote: post #325

Earlier quoted context omitted.

So your "goal" is to find an existing ai-auto completion that allows you to draft with AI then "write it yourself" by hitting tab? Sounds like the goal is actually to build that, then use it on projects....

I get pretty good results by writing specs and prototype with a LLM, through more or less managed conversations. But once the prototype is done, I spend too much time refining the details, fixing everything going wrong (bas design details, wrong implementation, half done testing ...) A full agentic setting would be too expensive for me (I wonder how much Garry tan spends...) So I'd like to take a more balanced approa…

Cursor with its tab completion. Iterate with the agent part to research, design, and plan. Let it generate boilerplate and scaffolding, perhaps with placeholders for you to fill in. Then fill in as you normally would, but with an auto-complete that uses all of your code and design docs and everything else to inform that completion rather than the limited set of info that shows up in an LSP.

I believe JetBrains IDEs have something similar too, but I don't have as much experience using theirs since my employer hasn't blessed their AI tools yet.

Re: Using AI to write better code more slowly

#487

Post by ku1ik »

throwaway7783 wrote: post #425
cold_harbor wrote: post #303

Earlier quoted context omitted.

LLMs flip positions when users push back ~70% of the time even when they were right. RLHF optimizes for approval, not correctness

Obviously this is just my experience. Claude code pushes back much harder than Codex.

I have totally opposite experience.

Re: Using AI to write better code more slowly

#488

Post by ptsneves »

yread wrote: post #194
throwaway7783 wrote: post #141

Earlier quoted context omitted.

I follow the same process. I have a design in mind for the problem at hand, but I don't reveal it to Codex. I go back and forth a bit to see if its proposals are better than mine. I go back and forth on tradeoffs of various approaches. And then I ask it to compare its proposals with mine. I "win" most of the time but there are many times where it shows a me a better, or simpler approach, or makes me rethink the solut…

> I go back and forth a bit to see if its proposals are better than mine I find it useful to let it generate benchmarks comparing the approaches. Turns out AI is terrible at guessing whats faster or allocates less

I had the exact experience yesterday.

I have a performance problem and went down the path of optimising part of a pipeline that when benchmarked was not the bottleneck, even if it looked plausible for me and the llm. When I asked it to make a final benchmark for documentation I found most of the work I did improved 30% while another path would have improved a magnitude more.

Thankfully iteration is now faster than ever and given how fast it creates tests, previous tests created for the aborted optimisation were helpful.

Re: Using AI to write better code more slowly

#489

Post by comboy »

bottlepalm wrote: post #419
comboy wrote: post #297

Earlier quoted context omitted.

Have you tried telling claude to review with subagent? It too almost always finds corner cases (usually nothing serious, but most stuff is things that good coder would have thought of)

How does that work? Isn't writing code and reviewing code things that happen in serial?

It implements something, and then subagent reviews it (the main one waits or does something else in the meantime, so yes it's serial, subagent is for having fresh context and evaluating changes as they stand not being biased by existing thinking and discussion), and then the main one implements fixes that are worth doing (because you will inevitably end up with some nitpicks).
Post reply on HN