Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI has a deep understanding of how this code works

github.com

Post reply on HN
Print view View original

161–170 of 329 posts

Re: AI has a deep understanding of how this code works

#161

Post by kaufmae »

ptsneves wrote: post #92
paxys wrote: post #78

Even if you are okay with AI generated code in the PR, the fact that the community is taking time to engage with the author and asking reasonable questions/offering reasonable feedback and the author is simply copy-pasting walls of AI-generated text in response warrants an instant ban. If you want to behave like a spam bot don't complain when people treat you like a spam bot.

Sometime ago I had a co-worker do this to me, pasting answers to my questions. He would paste the jira ticket to the ChatGPT(this was GPT3 time) and submit the PR. I would review it and ask questions and the answers had this typical rephrasing and persona of chatgpt. I had no proof, so one day i just used the PR and my comments as a prompt. The answers the co-worker gave me were almost the same down to the word as wh…

This. Choose your destiny: 1. Take time to review the code, post it to the author with knowing that nobody and nothing is going to learn from it except for you doing his job for feeding new prompts 2. Take ownership of the branch and fix the AI code 3. Read through the code to get some learning out of it if possible, close the PR and write your own

Re: AI has a deep understanding of how this code works

#162

Post by menaerus »

nsagent wrote: post #138
jodrellblank wrote: post #131

Earlier quoted context omitted.

> " It's clear some people have had their brain broken by the existence of AI. " The AI wrote code which worked, for a problem the submitter had, which had not been solved by any human for a long time, and there is limited human developer time/interest/funding available for solving it. Dumping a mass of code (and work) onto maintainers without prior discussion is the problem[1]. If they had forked the repo, patched i…

It's clear Claude adapted code directly from the OxCaml implementation (the PR author said he pointed Claude at that code [1] and then provides a ChatGPT analysis [2] that really highlights the plagiarism, but ultimately comes to the conclusion that it isn't plagiarized). Either that highlights someone who is incompetent or they are willfully being blasé. Neither bodes well for contributing code while respecting copy…

Sorry, this is just ridicilous and shows how people fragile really are. This whole topic and whole MR as well.

I am routinely looking into the folly implementation, sometimes into the libstdc++, sometimes into libc++, sometimes into boost or abseil etc. to find inspiration for problems that I tackle in other codebases. By the same standards, this should also be plagiarism, no? I manufacture new ideas by compiling existing knowledge from elsewhere. Literally every engineer in the world does the same. Why is AI any different?

Re: AI has a deep understanding of how this code works

#163

Post by Anamon »

xtracto wrote: post #98

This won't be a popular opinion here but, this resistance and skepticism of AI code, and people making it less smells to me very similar to the stance I see from some developers that have this belief that people from other countries CANNOT be as good as them (like, saying that outsourcing or hiring people from developing countries will invariably bring low[er] quality code). Feels a.but like snobbism and projection o…

This would be fine if LLMs generated quality code, which they don't. Anything beyond trivial and boilerplate code is either riddled with errors or copied almost verbatim. None of these systems are able to even remotely do what a competent developer does.

Despite the PR author's claims, LLMs have no, and can't have any, understanding of the code. Especially when you start talking about architecture, robustness, security, etc. And those are the really challenging parts. Coding is 10% of a developer's job, and they're usually the easiest. If reasonably used LLM tools can help developers code, awesome. But that part was never the problem or the bottleneck.

The chess/Go analogy doesn't work, because those are games that have set rules and winning conditions. Algorithms can work with that, that's why they beat humans. The "winning conditions" of software development are notoriously tricky to get right an often impossible to perfectly formulate. If they weren't, natural language programming might be a viable path. Dijkstra knew in the 70s that it can't be.[1]

Generated code can already not be ignored, but I don't think it's for the reasons implied. Someone here mentioned Brandolini's Law. We can't ignore it for the same reason we can't ignore spam e-mails. They're too easy and cheap to produce, and practically none of what's produced has any real value or quality. We can't ignore the code because it's threatening to make an already worrying crisis of QA and security in software development even worse.

[1] https://www.cs.utexas.edu/~EWD/transcriptions/EWD06xx/EWD667...

Re: AI has a deep understanding of how this code works

#164

Post by Anamon »

williamdclt wrote: post #88
djoldman wrote: post #5

Maintainers and repo owners will get where they want to go the fastest by not referring to what/who "generated" code in a PR. Discussions about AI/LLM code being a problem solely because AI/LLM is not generally a productive conversation. Better is to critique the actual PR itself. For example, needs more tests, needs to be broken up, doesn't follow our protocols for merging/docs, etc. Additionally, if there isn't a c…

> Better is to critique the actual PR itself. For example, needs more tests, needs to be broken up, doesn't follow our protocols for merging/docs, etc. They did: the main point being made is "I'm not reading 13k LOCs when there's been no proposal and discussion that this is something we might want, and how we might want to have it implemented". Which is an absolutely fair point (there's no other possible answer reall…

Exactly, this seems a bit overlooked in this discussion. A PR like this would NOT have been okay even if there was no LLM involved.

It reminds me of a PR I once saw (don't remember which project) in which a first-time contributor opened a PR rewriting the project's entire website in their favourite new framework. The maintainers calmly replied to the effect of, before putting in the work, it might have been best to quickly check if we even want this. The contributor liked the framework so much that I'm sure they believed it was an improvement. But it's the same tone-deafness I now see in many vibe coders who don't seem to understand that OSS projects involve other people and demand some level of consensus and respect.

Re: AI has a deep understanding of how this code works

#165

Post by torben-friis »

jodrellblank wrote: post #153
torben-friis wrote: post #139

Earlier quoted context omitted.

>They claim to have read the code, tested the code, they know that other people want the functionality; is wanting to share working code a "broken brain"? There is clearly a deviation between the amount of oversight the author thinks they provided and the actual level of oversight. This is clear by the fact that they couldn’t even explain the misattribution. They also mention that this is not their area of expertise.…

If a manager says they provided oversight of their developer employees, and the code was not as good as the manager thought, would you say "the manager has had their brain broken by the existence of employees"?

I'll bite, let's grant for the sake of the argument that equaling the LLM with a person holds.

This manager is directly providing an unrelated team with an unprompted 150-file giant PR dumped at once with no previous discussion. Upon questioning, he says the code has been written by an outside contractor he personally chose.

No one has onboarded this contractor to the team, and checking their online presence shows lots of media appearances, but not a single production project in their CV, much less long time maintenance.

A cursory glance at the files reveals that the code contains copypasted code from stackoverflow to the point that the original author's name is still pasted in comments. The manager can not justify this, but doesn't seem bothered by the fact, and insists that the contractors are amazing because he's been following them in social networks and is infatuated with their media there.

Furthermore, you check the manager's history in slack and you see 15 threads of him doing the same for other teams. The ones that have agreed to review their PRs have closed them for being senseless.

How would you be interacting with this guy?

Re: AI has a deep understanding of how this code works

#166

Post by KurSix »

This is a perfect real-world illustration of Brandolini's law: the amount of energy needed to refute bullshit is an order of magnitude bigger than to produce it.

The guy spent 5 minutes prompting, while Oсaml maintainers spent hours of their time politely dissecting the mess. Open Source will lose this war unless it changes the rules of engagement for contributions

Re: AI has a deep understanding of how this code works

#167

Post by KurSix »

xtracto wrote: post #98

This won't be a popular opinion here but, this resistance and skepticism of AI code, and people making it less smells to me very similar to the stance I see from some developers that have this belief that people from other countries CANNOT be as good as them (like, saying that outsourcing or hiring people from developing countries will invariably bring low[er] quality code). Feels a.but like snobbism and projection o…

The chess analogy is fundamentally flawed. In chess you don't have to maintain your moves - you make a move, and it's done. In engineering code isn't the end of the game, it's the start of a liability.

Code is read 10x more often than it is written. A programmer's primary job isn't "making the computer do X," but "explaining to other programmers (and their future self) why the computer should do X." AI generates syntax, but it lacks intent.

Refusing to accept such code isn't snobbery or fear. It's a refusal to take ownership of an asset that has lost its documentation regarding provenance and meaning

Re: AI has a deep understanding of how this code works

#168

Post by mnming »

benterix wrote: post #62

Did these Ocaml maintainers undergo some special course for dealing with difficult people? They show enormous amounts of maturity and patience. I'd just give the offender Torvalds' treatment and block them from the repo, case closed.

I wonder if it's the best outcome? The contributor doesn't seem to have a bad intention, could his energy be redirected more constructively? E.g. encouraging him to split up the PR, make a design proposal etc.

Re: AI has a deep understanding of how this code works

#169

Post by maleldil »

nikcub wrote: post #31

https://github.com/ocaml/ocaml/pull/14369/files#diff-bc37d03... Found this part hilarious - git ignoring all of the claude planning MD files that it tends to spit out, and including that in the PR Lazy AI-driven contributions like this are why so many open source maintainers have a negative reaction to any AI-generated code

The AI should've told him that you can have a local gitignore (.git/info/exclude)

Re: AI has a deep understanding of how this code works

#170

Post by yodsanklai »

benterix wrote: post #62

Did these Ocaml maintainers undergo some special course for dealing with difficult people? They show enormous amounts of maturity and patience. I'd just give the offender Torvalds' treatment and block them from the repo, case closed.

In my big tech company, you don't want to be dismissive of AI if you don't want to sound like a paria. It's hard to believe how much faith leadership has in AI. They really want every engineer to use AI as much as possible. Reviewing is increasingly done by AI as well.

That being said, I don't think that's why reviewers here were so cordial, but this is the tone you'd expect in the corporate world.

Post reply on HN