Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Zig project's rationale for their anti-AI contribution policy

simonwillison.net

Post reply on HN
Print view View original

321–330 of 496 posts

Re: The Zig project's rationale for their anti-AI contribution policy

#321

Post by sauercrowd »

zeeveener wrote: post #313
branko_d wrote: post #91

From https://kristoff.it/blog/contributor-poker-and-ai/ : "Unfortunately the reality of LLM-based contributions has been mostly negative for us, from an increase in background noise due to worthless drive-by PRs full of hallucinations (that wouldn’t even compile, let alone pass CI), to insane 10 thousand line long first time PRs. In-between we also received plenty of PRs that looked fine on the surface, some of which…

I'm personally amazed that _Large_ OSS projects don't have the appropriate automation in place to prevent non-compiling or non-linter-passing submissions. - Hooks (although there's no clean way to enforce they be "installed" on a clone), GHA Workflows (or their equivalents on other forges). This might be my bias showing, but these are items I would consider table-stakes for a project of a certain size / level of popu…

I mean even having linters and everything still creates a whole bunch of noise in their PR section, not to mention that a lot of the changes I make to stuff that's written by codex is not stuff that's caught by linters.

It's just bad/wrong/context lacking decisions and mental models it introduces, that if not carefull will just create a massive mess of a codebase. (I know, because I've tried, and had to deal with it)

And if someone vibecodes a PR and it works, why dont they just share the prompt so a repo owner could vibecode it themselves?

Re: The Zig project's rationale for their anti-AI contribution policy

#322

Post by abustamam »

zeeveener wrote: post #313
branko_d wrote: post #91

From https://kristoff.it/blog/contributor-poker-and-ai/ : "Unfortunately the reality of LLM-based contributions has been mostly negative for us, from an increase in background noise due to worthless drive-by PRs full of hallucinations (that wouldn’t even compile, let alone pass CI), to insane 10 thousand line long first time PRs. In-between we also received plenty of PRs that looked fine on the surface, some of which…

I'm personally amazed that _Large_ OSS projects don't have the appropriate automation in place to prevent non-compiling or non-linter-passing submissions. - Hooks (although there's no clean way to enforce they be "installed" on a clone), GHA Workflows (or their equivalents on other forges). This might be my bias showing, but these are items I would consider table-stakes for a project of a certain size / level of popu…

All of my personal projects, many of which will never be publicized, use hooks and GHA to ensure compilation of changes.

It is quite strange that a large project like Zig would not have such a thing. I'm sure it's not trivial but it seems important to invest time into.

Re: The Zig project's rationale for their anti-AI contribution policy

#323

Post by papyrus9244 »

zeeveener wrote: post #313
branko_d wrote: post #91

From https://kristoff.it/blog/contributor-poker-and-ai/ : "Unfortunately the reality of LLM-based contributions has been mostly negative for us, from an increase in background noise due to worthless drive-by PRs full of hallucinations (that wouldn’t even compile, let alone pass CI), to insane 10 thousand line long first time PRs. In-between we also received plenty of PRs that looked fine on the surface, some of which…

I'm personally amazed that _Large_ OSS projects don't have the appropriate automation in place to prevent non-compiling or non-linter-passing submissions. - Hooks (although there's no clean way to enforce they be "installed" on a clone), GHA Workflows (or their equivalents on other forges). This might be my bias showing, but these are items I would consider table-stakes for a project of a certain size / level of popu…

One of my pet peeves with git (and systems both similar, and based on it) is that automated tests run after you've made the commit and push.

In my mind the commit (let alone the push to a publicly accesible server) should be done after, and only if, the automated tests are successfully executed. And there's no easy way to implement this, other than having a dirty branch that you discard after rebasing onto a more long lived one.

Re: The Zig project's rationale for their anti-AI contribution policy

#324

Post by pxc »

zeeveener wrote: post #313
branko_d wrote: post #91

From https://kristoff.it/blog/contributor-poker-and-ai/ : "Unfortunately the reality of LLM-based contributions has been mostly negative for us, from an increase in background noise due to worthless drive-by PRs full of hallucinations (that wouldn’t even compile, let alone pass CI), to insane 10 thousand line long first time PRs. In-between we also received plenty of PRs that looked fine on the surface, some of which…

I'm personally amazed that _Large_ OSS projects don't have the appropriate automation in place to prevent non-compiling or non-linter-passing submissions. - Hooks (although there's no clean way to enforce they be "installed" on a clone), GHA Workflows (or their equivalents on other forges). This might be my bias showing, but these are items I would consider table-stakes for a project of a certain size / level of popu…

> Hooks (although there's no clean way to enforce they be "installed" on a clone), GHA Workflows (or their equivalents on other forges).

Git supports pre-receive hooks. But big multitenant forges like GitHub.com don't allow you to configure them because they're difficult to secure well. (Some of their commercial features are likely based on them, though.)

If you self-host a forge, though, you can configure arbitrary pre-receive hooks for it in order to do things like prevent pushes from succeeding if they contain verifiably working secrets, for example. You could extend that to do whatever you want (at your own risk).

Re: The Zig project's rationale for their anti-AI contribution policy

#325

Post by umvi »

krupan wrote: post #300
nayroclade wrote: post #243

It seems like this policy will help them win at contributor poker in the short term, but lose in the end. The next generation of developers will, for better or worse, grow up using AI assistance to write their code, but none of them will ever become a Zig contributor.

I still can't understand why people believe that this is the future. Especially for green field work like new compilers. LLMs do not invent new things. They cannot produce anything smarter/better than what they have been trained on. The big advantage they provide is producing (regurgitating) code faster than humans and better than less experienced/knowledgeable humans.

Ultimately code is an iterative refining process, like sculpting granite or spinning pottery. You start rough and iteratively shape and polish it. LLMs just rapidly speedup the iterative process. The next generation will be using LLMs to quickly setup the rough shape of new software and then iteratively refine them.

The "smarter/better" attributes you are worried about LLMs not having happen between iterative steps, when the human is inspecting the current state of the software and compares it to the desired state of the software (in their mind's eye). The human then course corrects for the next iteration.

This would be like if Michelangelo carved the David using a robotic 6-axis chisel. It takes him 1 month instead of 3 years because he can convey his initial vision to the robot and then iteratively refine the granite until it matches his vision.

You can try to claim LLMs don't invent new things, but humans using LLMs absolutely invent new things (source: myself).

Re: The Zig project's rationale for their anti-AI contribution policy

#326

Post by vga1 »

em-bee wrote: post #304
vga1 wrote: post #121

Earlier quoted context omitted.

It seems to me that people might be arguing from conflicting hidden premises here. "AI Coding" is a spectrum that could mean something as simple as letting the LLM proofread your changes and then act on those with your own human brain, or it could mean just telling the agent what you want and let it rip and tear until it is done. If I do the latter and submit a PR to something like Zig, I'll be certainly caught doing…

Blanket banning all of these seems like a bad idea to me. It actively gates people like myself from contributing in my projects i will reject any contribution that i do not understand. even if the contribution is handwritten by an expert developer. that developer will have to earn my trust like anyone else, like you would have too. LLM contributions are non-deterministic, which means they can never be trusted. theref…

>because if you could understand it, then you could write it yourself.

I accept most things you said there as valid opinions, but this is where the logic goes wrong.

I use LLMs to give me more from the only resource (now that my basic and mid-level needs are largely met) that ultimately matters: time. That means that I need to waste far less time in front of the computer, typing code, and use far more time doing more useful things, like hobbies, art, being with my children.

But as I said before, every project is obviously allowed to make their own rules, and contributors should obey those rules. There are plenty of projects that take both AI deniers and plenty of projects who prefer AI aficiandos.

At least for now. My belief is that one those groups will fade away like horseback riding did, but we'll see. Perhaps you have heard the famous stages quoted by many different people in different forms: first an idea is ridiculed, then it's attacked, then it's accepted. Some open-source communities have clearly entered the attacking phase in the last year so.

Re: The Zig project's rationale for their anti-AI contribution policy

#327

Post by debarshri »

We have been running LLM and coding agents for a while now and my overall observation is that it is a powertool or a crane, it is not a decision making tool.

Now in my org, people who have great understanding of concepts, deeper engineering understand have exponential productivity. People who dont or new in the workforce, juniors, are generating hell-ish code without understand as long as it runs they think the job is done. And this is where the problem is.

The llm creates an intellectual gap within the org and it just widens it as more and more it gets used. You might end up not trusting stuff within the org if code is generated by later.

Re: The Zig project's rationale for their anti-AI contribution policy

#328

Post by abustamam »

sauercrowd wrote: post #321
zeeveener wrote: post #313

Earlier quoted context omitted.

I'm personally amazed that _Large_ OSS projects don't have the appropriate automation in place to prevent non-compiling or non-linter-passing submissions. - Hooks (although there's no clean way to enforce they be "installed" on a clone), GHA Workflows (or their equivalents on other forges). This might be my bias showing, but these are items I would consider table-stakes for a project of a certain size / level of popu…

I mean even having linters and everything still creates a whole bunch of noise in their PR section, not to mention that a lot of the changes I make to stuff that's written by codex is not stuff that's caught by linters. It's just bad/wrong/context lacking decisions and mental models it introduces, that if not carefull will just create a massive mess of a codebase. (I know, because I've tried, and had to deal with it)…

Vibe coding is often not a single prompt, it's an entire workflow (if you're doing it right).

Re: The Zig project's rationale for their anti-AI contribution policy

#329

Post by notnullorvoid »

jart wrote: post #3

> This makes a lot of sense to me. It relates to an idea I've seen circulating elsewhere: if a PR was mostly written by an LLM, why should a project maintainer spend time reviewing and discussing that PR as opposed to firing up their own LLM to solve the same problem? The same argument applies to open source itself. Why use someone's project when you can just have the robot write your own? It's especially true if the…

You are missing the point of the original argument.

It's not that the project maintainer can use a LLM to generate a PR, it's that they choose not to.

To relate it closer to your argument. As a someone involved in a project that does X, I would find little value in collaborating with the "author" of another project built with AI to do X. Where as a project doing X were the authors actually wrote, understand the code, and thus the problem space better would be extremely valuable peers.

Post reply on HN