201–206 of 206 posts
Earlier quoted context omitted.
Wait and see, then change the policy based on what actually happens. I sort of doubt that all of a sudden there's going to be tons of people wanting to make complex AI contributions to LLVM, but if there are just ban them at that point.
It has happend to Curl.
Earlier quoted context omitted.
> I've been using AI to contribute to LLVM, which has a liberal policy. This is a different decision made by the LLVM project than the one made by Gentoo, which is neither right nor wrong IMHO. > The code is of terrible quality and I am at 100+ comments on my latest PR. This may be part of the justification of the published Gentoo policy. I am not a maintainer of same so cannot say for certain. I can say it is implie…
I would also ask - of the requested changes reviewers have made, what percentage are due to LLM generated changes? If more than zero, does this corroborate the Gentoo policy position of "Popular LLMs are really great at generating plausibly looking, but meaningless content." I can only speak for my own PR, but most requested changes were related to formatting and other stylistic issues that I didn't fully grasp as a…
> However, there was plenty of erroneous and invalid behaviour in the original AI-generated code ...
> I prevented most of this from reaching the PR by writing good unit tests and having a clear vision of what the final result should look like.
This identifies an interesting question in my mind:
If an LLM code generator is used, is it better to use
it for generating production code and writing tests
to verify or write production code and use LLM
generated code to produce tests to verify?
Maybe this question would be a good candidate for an "Ask HN".
> I believe this should be a basic requirement for trying to contribute AI-generated code to an open-source project but other people might not share the same belief.
FWIW, I completely concur.
Earlier quoted context omitted.
I've been using AI to contribute to LLVM, which has a liberal policy. The code is of terrible quality and I am at 100+ comments on my latest PR. That being said, my latest PR is my second-ever to LLVM and is an entire linter check. I am learning far more about compilers at a much faster pace than if I took the "normal route" of tiny bugfixes. I also try to do review passes on my own code before asking for code review…
[flagged]
Be kind. Don't be snarky. Converse curiously; don't cross-examine. Edit out swipes.
When disagreeing, please reply to the argument instead of calling names. "That is idiotic; 1 + 1 is 2, not 3" can be shortened to "1 + 1 is 2, not 3."
Please don't fulminate. Please don't sneer, including at the rest of the community.
Please respond to the strongest plausible interpretation of what someone says, not a weaker one that's easier to criticize. Assume good faith.
Please don't post shallow dismissals, especially of other people's work. A good critical comment teaches us something.
Earlier quoted context omitted.
The linked API policy lists specific concerns in 3 categories: copyright, quality, ethical. Which one do you not understand?
I don't care about "ethics" in the abstract if the code works and is of good "quality" (however you choose to define that). AIs don't have copyright over anything they generate, so that's a non issue. In fact, if the code is any good, it should be impossible to tell if it was written by AI at all.
But there are people who do care.
> AIs don't have copyright over anything they generate, so that's a non issue.
The problem is AIs also don't have copyright for the material they are trained on.
> In fact, if the code is any good, it should be impossible to tell if it was written by AI at all.
Do you mean, if somebody can successfully hide the fact they used AI, all concerns go away because nobody will be able to tell anyway?
Perhaps the most telling portion of their decision is: Quality concerns. Popular LLMs are really great at generating plausibly looking, but meaningless content. They are capable of providing good assistance if you are careful enough, but we can't really rely on that. At this point, they pose both the risk of lowering the quality of Gentoo projects, and of requiring an unfair human effort from developers and users to…
I've been using AI to contribute to LLVM, which has a liberal policy. The code is of terrible quality and I am at 100+ comments on my latest PR. That being said, my latest PR is my second-ever to LLVM and is an entire linter check. I am learning far more about compilers at a much faster pace than if I took the "normal route" of tiny bugfixes. I also try to do review passes on my own code before asking for code review…
Earlier quoted context omitted.
I would also ask - of the requested changes reviewers have made, what percentage are due to LLM generated changes? If more than zero, does this corroborate the Gentoo policy position of "Popular LLMs are really great at generating plausibly looking, but meaningless content." I can only speak for my own PR, but most requested changes were related to formatting and other stylistic issues that I didn't fully grasp as a…
Thank you for sharing your experiences in using this approach. They are ones which cannot be ascertained from PR's alone. > However, there was plenty of erroneous and invalid behaviour in the original AI-generated code ... > I prevented most of this from reaching the PR by writing good unit tests and having a clear vision of what the final result should look like. This identifies an interesting question in my mind: I…
> If an LLM code generator is used, is it better to use it for generating production code and writing tests to verify or write production code and use LLM generated code to produce tests to verify?
I do both.
1. Vibe code the initial design with input on the API/architecture.
2. Use the AI to write tests.
3. Carefully scrutinize the test cases, which are much easier to review than the code.
4. Save both.
5. Go do something else and let the AI modify the code until the tests/linting/etc passes.
6. Review the final product, make edits, and create the PR.
The output of step 1 is guaranteed to be terrible/buggy and difficult to review for correctness, which is why I review the test cases instead because they provide concrete examples.
Step 5 eliminates most of the problems and frees me to review important stuff.
The whole reason I wrote the check is because AI keeps using `int` and I don't want it to.