Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

When I reject AI code even if it works

vinibrasil.com

Post reply on HN
Print view View original

111–120 of 184 posts

Re: When I reject AI code even if it works

#111

Post by Planktonne »

lemagedurage wrote: post #88
gib444 wrote: post #81

Earlier quoted context omitted.

This idea is being pushed to increase sunk costs IMO. We are told to spend huge amount of time writing specs, behaviour tests, AGENTS.md and prompts. Pinky promise that's enough to get good output. Pinky promise we won't invent yet another body of work the whole industry must adopt to get good output. Pinky promise the AI tool will properly read all your work And then of course we are told you must never trust its ou…

Wow hang on, I'm suggesting to use AI as a code writing aid, not to increase scope until owning the design becomes unreasonable.

It's been years at this point though; everyone knows where "use AI as a code writing aid" ends up.

Re: When I reject AI code even if it works

#112

Post by ffsm8 »

abhgh wrote: post #25
Aurornis wrote: post #10

Even using Fable (while it was briefly available), having it refine a plan, and directing it to make only small incremental changes, I still found reasons to reject its first pass at a lot of work. There was a lot of “You’re right to push back” responses. A lot of incidents where it would creat some giant complex set of abstractions to accomplish something that I could find ways to do much more elegantly and in a mor…

These "You're right to push back" scenarios are scary for me. I mostly code ML implementations, and some of the errors Claude Code (CC - have only used Opus 4.7) makes are very sneaky, and if you don't have sufficient experience in the area (I see this with people entering ML and writing their implementations with CC), you wouldn't know when to question CC and will let errors or future pitfalls silently slip into you…

What's mind-blowing to me is that people see the "you're right to push back" as anything besides hallucination / self affirmation

Dude, the fucking model is great for sure, but there is nothing behind the illusion. It doesn't know if something is right or wrong - simpler or harder to reason about etc

It's just generating text, in a coherent manner while following rhetoric processes as a solid attempt at logical thinking

Why is that so hard for people to grok?

Our industry (and society after) is beyond doomed with people seeing these self affirmations as anything like "insightful" validation.

Re: When I reject AI code even if it works

#113

Post by danaris »

embedding-shape wrote: post #15
Aurornis wrote: post #10

Even using Fable (while it was briefly available), having it refine a plan, and directing it to make only small incremental changes, I still found reasons to reject its first pass at a lot of work. There was a lot of “You’re right to push back” responses. A lot of incidents where it would creat some giant complex set of abstractions to accomplish something that I could find ways to do much more elegantly and in a mor…

> There are a lot of workplaces where there isn’t a good mechanism to push back on this and the tech debt just keeps growing. If the "big ball of spaghetti" theory holds, where software companies who can't manage the debt stumble over themselves as they continue to add to the big ball of spaghetti code, I guess we'll see a row of companies declaring "software bankruptcy" or something in some/many months, depending on…

The problem is that this is just another instance of trusting that "the market will solve all our problems."

But that's based on "spherical economy in a frictionless vacuum" type assumptions.

In the real world, in addition to the problems others have noted of it being hard to identify and fix the specific sources of problems, we have so much consolidation that it doesn't matter if something from any of the tech giants starts getting buggier and slower. What are you* going to do—switch from Windows to Linux, just because it's getting a bit buggy? Or worse, switch away from Banner, or Salesforce?

We cannot depend on "market forces" to prove whether LLM-assisted coding is actually a good idea. We have to push for universal personal accountability for the code we commit (at least internally; I'm not calling for legal liability here!). Which is, unquestionably, going to be a huge uphill slog.

* where "you" in this case is an average PC user, or a large institution

Re: When I reject AI code even if it works

#114

Post by danaris »

kerkeslager wrote: post #74
darkerside wrote: post #33

Earlier quoted context omitted.

In fairness, you could throw the most senior engineer into a brand new codebase, and they would probably make a dozen mistakes if you immediately had them pick up invasive and risky work.

No, that's not "in fairness", that's misunderstanding the entire problem. Having worked 20 years in this field and managed a few projects, no, I wouldn't make a dozen mistakes, because I would refuse to take on work I can't responsibly do. Invasive and risky work IS the thing I want to be working on because it's the place where I can be most valuable, but part of my value comes from asking the right people the right…

> I'm going to work directly with the people who wrote it, and only when THEY think I understand it well enough am I venturing in alone.

...ah, what a boon it would be to be working with code written by people still working at the organization!

(No shade, just being wistful; I happen to have a history of coming in and having to deal with some messy codebases from the guy who just retired...)

Re: When I reject AI code even if it works

#115

Post by tgv »

jdw64 wrote: post #40

Coding with AI eventually comes down to two paths, I've realized. One is using AI exclusively for everything. The other is not using it at all. There is almost no middle ground. The reason is that as the complexity and depth of the problem increase, the code AI generates increasingly follows enterprise level patterns. The deeper the meaning of what I input, the more AI tends to produce code that goes beyond my own ar…

> There is almost no middle ground.

I use it rarely. I did have it rewrite some code, mainly from one language to another. That works really well. I also had it rewrite a database interface, which also seems to work (no time to test it thoroughly, yet, so it's not in production). But I'll be damned if I let it write new features. I've debugged other people's code, and it ain't fun. Debugging 10kLOC AI code sounds like hell to me.

Re: When I reject AI code even if it works

#116

Post by lemagedurage »

Planktonne wrote: post #111
lemagedurage wrote: post #88

Earlier quoted context omitted.

Wow hang on, I'm suggesting to use AI as a code writing aid, not to increase scope until owning the design becomes unreasonable.

It's been years at this point though; everyone knows where "use AI as a code writing aid" ends up.

I think the differentiator is whether someone cares about what they build or not. Someone who doesn't care wouldn't produce masterpieces without AI, and using AI isn't going to prevent someone who does care from building something nice.

Re: When I reject AI code even if it works

#118

Post by figassis »

Aurornis wrote: post #10

Even using Fable (while it was briefly available), having it refine a plan, and directing it to make only small incremental changes, I still found reasons to reject its first pass at a lot of work. There was a lot of “You’re right to push back” responses. A lot of incidents where it would creat some giant complex set of abstractions to accomplish something that I could find ways to do much more elegantly and in a mor…

Have nee dealing with this in an area that requires insane attention: payments. It's strange feeling when you architect a system, all the invariants, all the fundamentals, all the guardrails, then implement the scaffolding in self documenting code, so the LLM has no way to build other than correctly, but you then see what it tries to do and it's WTF.

It all seems to behave correctly and then you run your test suite, and your e2e tests start failing in weirs ways, a few but not many accounting discrepancies, and everything else passes. You spend a lot of time asking it to explain what's happening, you give it the data to browse, and it keeps giving you very plausible explanations of "found the issue, the data shows this clearly, there fore the bug is here, all I need to do is fix this thing", and it does this, and it still fails.

When you open the hood, man, the code salad, the 100s of unnecessary, and complex and duplicate abstractions, the stacked mistakes and lazy corrective attempts, the comment pollution that overrides your instructions across sessions.

You realize that there are things and concepts that it just cannot wrap it's "mind" around and you need to grab the wheel for a bit, make the corrections, remove all the comment litter, commit and then hand the wheel back and tell it to "look at the last commit so see what I mean. explain to me what you did wrong and update all documentation, memory and context with this new understanding".

So if you have no experience in the field, you won't even know how to test, how to find that there is an issue, the appearance of "working" and the AI's confidence will trip you in prod so hard.

Re: When I reject AI code even if it works

#119

Post by CraigJPerry »

The bottleneck when using a "faster keyboard" is understanding. We have a tool for this in compsci. Not having to fully understand something in order to successfully exploit it is a staple of computer science; we use abstractions to help us reason at a higher level. You don't necessarily always have to understand the nuance involved in selecting a hash function just to put and get some items in a hash map. Specifically, when are these cases where you don't need to go that deep? Are there similar scenarios for ai written code?

I'm more interested right now in what does that abstraction look like for AI generated code. Is there some reasonable solution wherein a sandboxed component in the enterprise architecture has various attributes (e.g. the bytes i stuff into this file store component are always the exact bytes i get back from it) confirmed by methods other than a human reading its code? Those methods, are they cheaper, faster, safer than just having a human do it?

If your enterprise architects have to read every line of code in your system today then i'd claim your architecture practices have room to mature. What can derived from that, and in which scenarios, for the purposes of safely leveraging immutable write-only code? I'm not interested in evolving the code (lines of code spent to solve a business problem was never an asset, it was always a cost) if it wasn't hand crafted by a human, i still have the requirements so i can just regenerate the entire thing with the revised requirement.

Re: When I reject AI code even if it works

#120

Post by tom2026hn »

SunboX wrote: post #99

I unterstand the reasons, but I don't think so. I have experience in software development over 20 years now and still developing software daily. Nowadays it's nearly 100℅ AI written. It looks good and works. Sure, you have to guide the AI. But this can be done with custom skills, angent files, code quality guards test cases and so on. Maybe the code looks at the end not as I would have written it, maybe something is…

Do you still need to review the code? LLMs generate code too quickly for humans to keep up.
Post reply on HN