Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I'm going back to writing code by hand

blog.k10s.dev

Post reply on HN
Print view View original

631–640 of 656 posts

Re: I'm going back to writing code by hand

#631

Post by pramodbiligiri »

haolez wrote: post #387

I've started using OpenSpec[0] recently to mitigate problems like that, but I'm still very early in this journey. Can someone with more experience with it (or similar tools) chime in and confirm that this isn't just more AI snake oil? :) [0] https://openspec.dev/

Some kind of planning / speccing out is becoming inevitable. No personal experience with openspec but I do rely on generating plans, and then a set of tasks from the plan. And keeping a close eye on what's going as the tasks are churned through (although I wonder if simply saying Yes to the diffs has been adding much value /shrug).

Matt Pocock talks about specs and Openspec after 23:00 minute mark and again after 33:00 minute mark here: https://www.youtube.com/watch?v=-QFHIoCo-Ko. He doesn't believe in simply translating specs-to-code. He emphasizes tracer bullets, TDD, setting up quick feedback loops.

Re: I'm going back to writing code by hand

#632

Post by pron »

K0balt wrote: post #603
pron wrote: post #308

Earlier quoted context omitted.

Sure, if you carefully review the agent's output, including tests, you can get good results. If you don't carefully review the output, you obviously have no idea if it's good enough for you. The only way to find out is that 30 changes down the line the agent won't be able to change one thing without breaking another, but by then the codebase will be too far gone to fix.

This is essentially true. There are other ways to achieve this goal though, that don’t require exhaustive human review, better models are able to do that part as well if properly guided. The key is that yes, some of the design constraints will morph over time, necessarily, since coding is as often about discovering the problem as solving it. But design principles don’t drift. If you have a design principle that can n…

> There are other ways to achieve this goal though, that don’t require exhaustive human review, better models are able to do that part as well if properly guided.

Not at this time. Even if you could somehow get their success rate to 90%, it's still far too low because the mistakes can be (and are occassionally) catastrophic. It's only when you review everything that you find mistakes that will bite you down the line. If you don't review everything, you just don't know, but the rate of bad mistakes introduced by the agents is too high to trust, no matter how much prompting and orchestration you do. Maybe future models will address that, but we're not there yet.

> The main thing that helps me in my workflow is to develop documentation around the code. If the code drifts from the docs, the model will notice and you can decide which was correct, the plan, the maintainer manual, or the code, or the comments in the code.

That's helpful but it doesn't solve the problem, which is that the agents are happy to introduce horrendous workarounds, and they don't tell you that the code they've written is a horrendous workaround. The docs are fine and reflect the code and the code reflects the strategy, but you just don't know that the strategy is wrong.

Re: I'm going back to writing code by hand

#633

Post by tcgv »

rs999gti wrote: post #622
tcgv wrote: post #445

Earlier quoted context omitted.

> "Yep. The only people I've heard saying that generated code is fine are those who don't read it." I review every line of code I generate with AI. I mainly use an MR-based approach: 1) Provide a tightly scoped technical spec to Codex as a task, and ask for 3x solutions. Usually at least one of them is on the right track, and it is better to ditch a solution that went in the wrong direction than to try to fix it. 2)…

> Provide a tightly scoped technical spec to Codex as a task, and ask for 3x solutions. I'm using a personal license and Codex. What does this cost to generate 3x solutions as a starting point? Even in simple coding I have been doing, I notice Codex will burn through my Open AI subscription rather fast.

I'm using a ChatGPT Plus subscription, and I only use Codex in the cloud [1]. With this setup, I have never hit the usage limit.

On my most active days, I integrate around a dozen fully reviewed and adjusted MRs into my codebase.

[1] https://chatgpt.com/codex/cloud

Re: I'm going back to writing code by hand

#634

Post by dkersten »

mountainriver wrote: post #623
dkersten wrote: post #608

Earlier quoted context omitted.

That doesn’t conflict with my statement. Before: One person writes the code (and likely understands it thoroughly), another person reviews the code to spot obvious mistakes or shortcomings. Now: AI writes the code, a person reviews it to spot obvious mistakes or shortcomings. In the before case, you have a person who has a deeper understanding of the code and in the AI case, you don’t, instead you have even more code…

>When a competent programmer is writing the code, the human written code tends to be higher quality too This is a great idea, but on average is deeply untrue. Far and away most programmers today write significantly worse code than LLMs. Also LLMs are fantastic at generating high level summaries and comments in code

> Far and away most programmers today write significantly worse code than LLMs

Your experience with LLMs do not match my own. Not to say that I haven’t experienced terrible human written code where I’ve wondered what the author could possibly have been thinking, but overall, I still find LLM written code to be on the poor side.

Like, the code itself is ok, but the wider picture reasoning and abstractions are bad. It also makes really dumb decisions far too often. Or doggedly shoehorns its first idea in no matter how badly it fits.

Re: I'm going back to writing code by hand

#635

Post by K0balt »

pron wrote: post #632
K0balt wrote: post #603

Earlier quoted context omitted.

This is essentially true. There are other ways to achieve this goal though, that don’t require exhaustive human review, better models are able to do that part as well if properly guided. The key is that yes, some of the design constraints will morph over time, necessarily, since coding is as often about discovering the problem as solving it. But design principles don’t drift. If you have a design principle that can n…

> There are other ways to achieve this goal though, that don’t require exhaustive human review, better models are able to do that part as well if properly guided. Not at this time. Even if you could somehow get their success rate to 90%, it's still far too low because the mistakes can be (and are occassionally) catastrophic. It's only when you review everything that you find mistakes that will bite you down the line.…

I haven’t had this problem. Maybe it’s because of the language I’m using (C++) or maybe it’s because of the strict enforcement of modularity and public vs private interfaces, etc that I use? Also, the code is tested against the hardware with every change. Idk if that’s why my experience has been different from yours or not.

My workflow also requires a discussion of the architecture and methodology of each addition or change, but honestly because we define the interfaces first, and each concern is given its own .c and .h file, it’s very hard to sneak something in without me noticing and calling it out. (Which does happen occasionally)

I suspect that file level granularity may be one of the keys. It never is actually working on more than a couple hundred lines of code at a time, plus interfaces of related files. I end up with a hundred files where I might have had 30 coding by hand, but it is actually easier to reason about the code for me as well, and the number of files is not an issue because of the automation. Total LOC is about the same as I would produce by hand for the same work, which means it’s actually writing less, due to the interface overhead, so I’m pretty stoked about that. The only real nightmare for humans is the long includes.

OTOH if I don’t do all of this it will definitely go off the rails and produce garbage.

I’ve been writing c (and c++) for almost 40 years, and although that doesn’t mean I’m any good, it does mean I have developed a keen sense of smell and highly sensitive olfactory PTSD.

With the right structured environment, a SOTA model with a suspicious seasoned dev holding its hand can be easier to manage and much more productive than a small team. Or, maybe I’ve just sucked so bad my whole life that I can’t tell the difference, but at any rate it works well enough to ship without nightmares, and less bugs and patching than I had before.

Edit:

I should mention that if bugs get tricky, like hardware idiosyncrasies and things like that, the model just goes nuts.if I handle it very very carefully so that it does not try to understand the problem, and I just have it poke the firmware with a stick from a distance enough times and from enough angles, as long as I have successfully prevented it from trying to figure out the problem (which is not as easy as it seems like it would be) it actually will usually nail it. If it starts to guess it’s usually best just to roll back the context and start over with the poking (I have a harness so it does direct hardware probes)

There seems to be an analog for this for non hardware related issues, but it’s harder to sus out when you should be telling it that you specifically do not want it to attempt to understand or solve the problem until you’ve rigged and tested all of the debug messaging.

Re: I'm going back to writing code by hand

#636

Post by beringela »

chamomeal wrote: post #32
whattheheckheck wrote: post #31

Earlier quoted context omitted.

I find that really hard to believe. It's not like curing cancer

No but it can be a rube goldberg machine of insanity

As per the diagram here: https://kevinsmall.dev/ai/rube-goldberg-risk-agentic-dev/

Re: I'm going back to writing code by hand

#637

Post by [deleted] »

sirwhinesalot wrote: post #277
IdiotSavage wrote: post #260

Earlier quoted context omitted.

So, basically you need to micro-manage it. Where are your 10x gains now? And is it fun to work like that?

This is actually what I do. I'm extremely picky about the code and force the LLM to rewrite it 1000x times until it is basically exactly what I want. You might be wondering what is the point when it would be faster for me to just write the code myself? I have ADHD and for whatever reason telling the LLM what to do instead of doing it myself bypasses the task avoidance patterns and/or focus problems I tend to suffer f…

[deleted]

Re: I'm going back to writing code by hand

#639

Post by lo_zamoyski »

marginalia_nu wrote: post #413
lo_zamoyski wrote: post #368

Earlier quoted context omitted.

That's a little simplistic and lacking in nuance. > To get beyond being a hack, you need to understand the entire stack, including the code that you didn't write, including both libraries, frameworks and the OS, and including the hardware, the networking layers, and so forth. I think maybe you overestimate your own knowledge here. It's one thing to understand general principles and design or to understand a contextua…

Well I mean I've built an internet search engine from scratch[1] and I'm making a living off this successfully enough to have completely left the wagie existence for the foreseeable future, so I think I at least kinda walk the talk. I'm far from the best at anything and make no claims toward knowing everything, but I do think I have reasonable breadth in my experience and work, and I don't think I could have built so…

But that's sort of my point: you're not walking the walk of others. You are a solo dev. You are working on a project that can be developed by a solo dev, both in terms of size and time frames. It is a greenfield project, so you have been there since the beginning and therefore have a lot of context and knowledge of rationale, history, motives, etc. that rarely anyone has in industry, and certainly not early on.

Re: I'm going back to writing code by hand

#640

Post by 20k »

zzzeek wrote: post #274
ZaoLahma wrote: post #168

Earlier quoted context omitted.

> There seems to be a strong bias where using AI feels like you're making a lot of progress very quickly, but compared to manual coding it often seems to be significantly slower in practice. This metric highly depends on who uses the AI to do what, where strong emphasis is on "who" and "what". In my line of work (software developer) the biggest time sinks are meetings where people need to align proposed solutions wit…

the venn diagram of people who say "AI produces useless results I have to 100% throw away" and people who say "I've never successfully delegated parts of large software development to junior programmers" is a circle

Junior programmers learn and provide a wide value, AI does not
Post reply on HN