Live data from Hacker News

AI has a deep understanding of how this code works

github.com

31–40 of 329 posts

Re: AI has a deep understanding of how this code works

#31
https://github.com/ocaml/ocaml/pull/14369/files#diff-bc37d03...

Found this part hilarious - git ignoring all of the claude planning MD files that it tends to spit out, and including that in the PR

Lazy AI-driven contributions like this are why so many open source maintainers have a negative reaction to any AI-generated code

Re: AI has a deep understanding of how this code works

#34
post #9

Can we please go back to "You have to make an account on our server to contribute or pull from the git?" One of the biggest problems is the fact that the public nature of Github means that fixes are worth "Faux Internet Points" and a bunch of doofuses at companies like Google made "social contribution" part of the dumbass employee evaluation process. Forcing a person to sign up would at least stop people who need "Fa…

Fully agree, luckily I don't maintain projects on GitHub anymore, but it used to be challenging long before LLMs. I had one fairly questionable contribution from someone who asked me to please merge it because their professor tasked them to build out a GitHub profile. I kinda see where the professor was coming from, but that wasn't the way. The contributor didn't really care about the project or improving it, they cared about doing what they were told, and the quality of the code and conversation followed from that.

There's many other kinds of questionable contributions. In my experience, the best ones are from people who actively use the thing, somewhat actively engage in the community (well, tickets), and try to improve the software for themselves or others. From my experience, GitHub encourages the bad kind, and the minor barriers to entry posed by almost any other contribution method largely deters them. As sad as that may be.

Re: AI has a deep understanding of how this code works

#35
post #19
post #13

Earlier quoted context omitted.

> doesn’t have a brain behind it that can be accountable for the code later. the submitter could also bail just as easily. Having an AI make the PR or not makes zero difference for this accountability. Ultimately, the maintainer pressing the merge button is accountable. What else would your value be as a maintainer, if all you did was a surface look, press merge, then find blame later when shit hits the fan?

If I had a magic wand I would wish for 2 parallel open source communities diverging from today. One path continues on the track it has always been on, human written and maintained. The other is fully on the AI track. Massive PRs with reviewers rubber stamping them. I’d love to see which track comes out ahead. Edit: in fact, perhaps there are open source projects already fully embracing AI authored contributions?

How would you define “ahead”?

Re: AI has a deep understanding of how this code works

#37
There are LLMs with more self-awareness than this guy.

Repeatedly using AI to answer questions about the legitimacy of commits from an AI, to people who are clearly skeptical is breathtakingly dense. At least they're open about it.

I did love the ~"I'll help maintain this trash mountain, but I'll need paying". Classy.

Re: AI has a deep understanding of how this code works

#39

Earlier quoted context omitted.

Pretty much. I guess it’s open source but it’s not in the spirit of open source contribution. Plus it puts the burden of reviewing the AI slop onto the project maintainers and the future maintenance is not the submitters problem. So you’ve generated lots of code using AI, nice work that’s faster for you but slower for everyone else around you.

Another consideration here that hits both sides at once is that the maintainers on the project are few. So while it could be a great burden pushing generated code on them for review, it also seems a great burden to get new features done in the first place. So it boils down to the choice of dealing with generated code for X feature, or not having X feature for a long time, if ever.

Their issue seemed to be the process. They're setup for a certain flow. Jamming that flow breaks it. Wouldn't matter if it were AI or a sudden surge of interested developers. So, it's not a question of accepting or not accepting AI generated code, but rather changing the process. That in itself is time-consuming and carries potential risk.

Re: AI has a deep understanding of how this code works

#40
OP’s code (at least plausibly) helped him. From https://github.com/ocaml/ocaml/pull/14369#issuecomment-35568...

> Damn, I can’t debug OCaml on my Mac because there’s no DWARF info…But, hey, there’s AI and it seems to one-shot fairly complex stuff in different languages, from just a Github issue…My needs are finally taken care of!

So I do believe using an LLM to generate a big feature like OP did can be very useful, so much that I’m expecting to see such cases more frequently soon. Perhaps in the future, everyone will be constantly generating big program/library extensions that are buggy except for their particular usecase, could be swapped with someone else’s non-public extensions that they generated for the same usecase, and must be re-generated each time the main program/library updates. And that’s OK, as long as the code generation doesn’t use too much energy or cause unforeseen problems. Even badly-written code is still useful when it works.

What’s probably not useful is submitting such code as a PR. Even if it works for its original use-case, it almost certainly still has bugs, and even ignoring bugs it adds tech debt (with bugs, the tech debt is significantly worse). Our code already depends on enough libraries that are complicated, buggy, and badly-written, to the extent that they slow development and make some feasible-sounding features infeasible; let’s not make it worse.

Post reply on HN