Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

When I reject AI code even if it works

vinibrasil.com

Post reply on HN
Print view View original

161–170 of 184 posts

Re: When I reject AI code even if it works

#161

Post by GroksBarnacles »

Obscurity4340 wrote: post #130
ffsm8 wrote: post #112

Earlier quoted context omitted.

What's mind-blowing to me is that people see the "you're right to push back" as anything besides hallucination / self affirmation Dude, the fucking model is great for sure, but there is nothing behind the illusion. It doesn't know if something is right or wrong - simpler or harder to reason about etc It's just generating text, in a coherent manner while following rhetoric processes as a solid attempt at logical think…

How does it correct itself then? I often will push back without giving it the way out and it often does find it

Are you questioning how LLMs work? It's not a mystery up for debate, it's an open, well known system, you can go learn it for yourself and see.

Re: When I reject AI code even if it works

#162

Post by Obscurity4340 »

ffsm8 wrote: post #156
Obscurity4340 wrote: post #130

Earlier quoted context omitted.

How does it correct itself then? I often will push back without giving it the way out and it often does find it

If you're fantasy was real, then how can you also have it correct itself from a passable solution to a dumbsterfire? That fundamentally wouldn't happen if it wasn't just an illusion. There is value in it for sure and I can use it to write a lot of simple code, which is 99.99% of enterprise software - but that's another topic.

The coding aspect is a great example of why I am skeptical of the claim they cant reason (in its own way).

Something that can write a correct code snippet or even larger program that accepts the correct input and provides the correct output and otherwise is consistent with the given spec is doing something substantially more than just autocomplete.

Re: When I reject AI code even if it works

#163

Post by ffsm8 »

Obscurity4340 wrote: post #162
ffsm8 wrote: post #156

Earlier quoted context omitted.

If you're fantasy was real, then how can you also have it correct itself from a passable solution to a dumbsterfire? That fundamentally wouldn't happen if it wasn't just an illusion. There is value in it for sure and I can use it to write a lot of simple code, which is 99.99% of enterprise software - but that's another topic.

The coding aspect is a great example of why I am skeptical of the claim they cant reason (in its own way). Something that can write a correct code snippet or even larger program that accepts the correct input and provides the correct output and otherwise is consistent with the given spec is doing something substantially more than just autocomplete.

I did say

> It's just generating text, in a coherent manner while following rhetoric processes as a solid attempt at logical thinking

So yeah, I do agree that they can make a very reasonable amount of reasoning. As a matter of fact, they reason about things better then an average Joe off the street ime.

That's entirely unrelated to what I said though, I think you misinterpreted/misunderstood what I wrote earlier.

They can make solid attempts at reasoning, its just not grounded in reality. It just applies these rhetoric processes to the current text - but it doesn't understand wherever it's actually correctly reasoned. Hence the answer "you're right to push back on this" is just the model being a sycophant. The sentence does not mean that anything of value has been communicated in either direction, and thinking that it has means the person in question is suffering from ai psychosis

Re: When I reject AI code even if it works

#164

Post by mdavid626 »

vnbrs wrote: post #157
mdavid626 wrote: post #140

The problem is if your coworkers don’t do it this way.

That’s why I think code reviews should be human first. That’s where we get the most value out of it. AI code reviews are still good though, but they should be treated as linters in your CI pipeline.

I haven’t seen any useful AI code review so far.

Re: When I reject AI code even if it works

#165

Post by kajman »

wwind123 wrote: post #70
kajman wrote: post #61

Earlier quoted context omitted.

I'm always curious when I see these stories. How long have you been doing this, for what sort of work, and was the codebase mature before you began working like this?

Yeah, this one is easy: I have been doing this for half a year. I have a couple of projects worked out this way, all green-field projects, code base grew from 0 to tens of thousand of lines each.

That is interesting. Half a year is not nothing and I expect it's harder to keep a project functioning when the base is vibe coded rather than having mature abstractions and architecture already.

I am still skeptical on this method's ability to deliver polished products though. I've kept an eye out on it in the OSS world and don't think I've seen big anything yet.

Re: When I reject AI code even if it works

#167

Post by 4fffs »

Aurornis wrote: post #10

Even using Fable (while it was briefly available), having it refine a plan, and directing it to make only small incremental changes, I still found reasons to reject its first pass at a lot of work. There was a lot of “You’re right to push back” responses. A lot of incidents where it would creat some giant complex set of abstractions to accomplish something that I could find ways to do much more elegantly and in a mor…

"It’s really eye opening to work with these tools on a codebase you know deeply because these problems are everywhere."

Same could be said w.r.t interacting with LLMs on stuff you are an expert on.

The thing is laborious, over-does it, slow and wasteful.

Re: When I reject AI code even if it works

#168

Post by QuiEgo »

Aurornis wrote: post #10

Even using Fable (while it was briefly available), having it refine a plan, and directing it to make only small incremental changes, I still found reasons to reject its first pass at a lot of work. There was a lot of “You’re right to push back” responses. A lot of incidents where it would creat some giant complex set of abstractions to accomplish something that I could find ways to do much more elegantly and in a mor…

To be fair, before AI I had my fair share of coworkers throwing stuff over the fence who only cared about closing tickets and collecting credit.

You all know the feeling: you see a code review from _that person_ and you know its gonna be a long day. And you know they are going to fight you every step of the way and say “but it works” when you leave a comment about their code being hard to maintain.

Re: When I reject AI code even if it works

#169

Post by QuiEgo »

resonious wrote: post #16
Aurornis wrote: post #10

Even using Fable (while it was briefly available), having it refine a plan, and directing it to make only small incremental changes, I still found reasons to reject its first pass at a lot of work. There was a lot of “You’re right to push back” responses. A lot of incidents where it would creat some giant complex set of abstractions to accomplish something that I could find ways to do much more elegantly and in a mor…

All Claude models are huge suck ups. The "you're absolutely right" meme is real even if that exact phrase doesn't show up as much anymore. I don't want to start a fight or anything but IME Codex has a bit more of a spine. If you point out something weird, it sometimes gives a good reason for it. Whereas Claude will always say "whoopsie you're right as always sir" even when it's me who missed something.

I’ve had this experience as well. I love Codex for doing code reviews, it takes a way more direct, less passive tone when calling out issues.

Re: When I reject AI code even if it works

#170

Post by QuiEgo »

whilenot-dev wrote: post #94

Titles like these make me always point out the obvious: A working state is the absolute minimum requirement for any code to be merged, isn't it? ...imagine to merge something even though you know that's not working. Besides, this post has nothing specific to code produced by an LLM, and placing AI in the stated reasons feels completely arbitrary, or is rather a fallacy of our times: - I reject [AI] code when I can’t…

I’ve had multiple people say “you don’t work on code anymore, that’s for the AI. You work a level of abstraction above that. As long as you prove it works through testing, the code doesn’t matter anymore. It’s like looking at the assembly the compiler spits out now - who cares?”

These are the people who spit out an incredible volume of code with AI, to the point reviews simply can’t keep up.

The last person who said this to me works in embedded, where we look at the assembly all the time. Scary.

Post reply on HN