Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLM policy?

github.com

Post reply on HN
Print view View original

81–90 of 145 posts

Re: LLM policy?

#81

Post by CGamesPlay »

cyphar wrote: post #72
CGamesPlay wrote: post #43

Earlier quoted context omitted.

I think it's disrespectful to throw bad code on somebody else, regardless of its provenance. And (as a professional who uses AI coding agents) I understand that LLM code is often bad code. But the OP isn't complaining about the deluge of bad code being submitted as PRs, they're complaining about LLM code being submitted as PRs. I've also been on the other side of this, receiving some spammy LLM-generated irrelevant "…

(OP here.) Well, we don't receive that many low-quality PRs in general (I opened this issue to discuss solutions before it becomes a real problem). Speaking personally, when it does happen I try to help mentor the person to improve their code or (in the case where the person isn't responsive) I sit down and make the improvements I would've made and explain why they were made as a comment in the PR. When it comes to L…

Thanks for the response here. I also read your follow-up on the Github issue. I definitely agree with you on the mentorship point. The "CPU quota allocation mismatch" issue felt particularly agitating to read from the point of view as a maintainer. Human being acting as a proxy for ChatGPT. Have you considered putting an "LLM usage disclosure" question on the issue/PR templates? Something like "To what extent were AI tools used to create this issue/PR? What validation steps did you perform to ensure accuracy?"

The "hard-line policy" would then shift from being "used LLM tools" to "lied on the LLM usage disclosure", and it feels a lot less like selective enforcement (from my perspective). Obviously it won't stop these spammy issues/PRs, but neither will a hard-line policy against all AI.

Re: LLM policy?

#83

Post by rester324 »

blutoot wrote: post #47

Since the cat is out of the bag (i.e. there's no stopping of LLM-generated code / PR / issues now that vibe-coding is fairly universally accessible), the only thing in our (especially those who are custodians of software systems and products) control is to invest a lot more in testing and evals. Just plain opposing LLM-generated content as a policy is a losing battle at this point..

> to invest a lot more in testing and evals

I don't think this will work. The same arguments could have been said to mitigate junior devs' work or short timeline/high stakes project technical debt and rarely ever anyone listened

Re: LLM policy?

#84

Post by bohrbohra »

dropbox_miner wrote: post #3

Curious to know if others are seeing a similar uptick in AI slop in issues or PRs for projects they are maintaining. If yes, how are you dealing with this? Some of the software that I maintain is critical to container ecosystem and I'm an extremely paranoid developer who starts investigating any github issue within a few minutes of it opening. Now, some of these AI slop github issues have a way to "gaslight" me into…

AI slop attacks on the cURL project : https://www.youtube.com/watch?v=6n2eDcRjSsk

Re: LLM policy?

#85

Post by rester324 »

NamlchakKhandro wrote: post #82

Wouldn't you provide LLM friendly guidance material in your repo instead of spending effort on a pointless endeavor to ban them ?

I think OP wanted to have a proof that the submitter can demonstrate understanding of both the system and the code in the PR. So how would a guidance solve that? Also OP didn't want to ban them I believe

Re: LLM policy?

#86

Post by jrochkind1 »

jmyeet wrote: post #14

Some years ago I read the Neal Stephenson book Anathem . SPOILERS: it has a version of the Internet called the Reticulum and one thing I remember is that it was filled with garbage. True information subtly changed multiple times until it was garbage. And there were agents to see through the garbage. I imagined this to be a neverending arms race. Honestly, this is kind of where I see LLM generated content going where…

Seems about right.

> I can see a future in our lifetimes where a significant amount of our capital expenditure and energy consumption is used, quite simply, to produce garbage.

If you squint your eyes right at the shelves at Target or in the Amazon delivery trucks, or honestly just look around you most anywhere, you may not have to wait for the future to see it.

Re: LLM policy?

#88

Post by muzani »

forgotoldacc wrote: post #33
somenameforme wrote: post #12

Earlier quoted context omitted.

In some ways I'm starting to enjoy this. Do you remember the 419 scams? 'Hi, I'm a Nigerian prince named Michael Jordan. Give me $50 so I can buy some chemicals to clean a bunch of money I secretly stowed away and I'll send you $5000.' People actually used to fall for that. Of course some people probably still would (and a lot more certainly gets blocked by spam blockers) but I think overall society grew less substan…

> but I think overall society grew less substantially less gullible over time. Absolutely not. All that happened is most people became aware that "Nigerians offering you money are scammers." But they still fall for other get rich quick schemes so long as it diverges a little bit from that known pattern, and they'll confidently walk into the scam despite being warned and saying, "It's not a scam, dumbass. It's not lik…

I call it the antibody effect. My favorite example is clickbait headlines like, "Five things you MUST do if you're doing this thing. You'd never guess #3!" It used to be everywhere and now it's nowhere.

AI is starting to show this effect - people stay away from em-dashes. There's that yellowish tinge and that composition which people avoid on art. Some of this is bad, but we can probably live without it.

Re: LLM policy?

#89

Post by timedrun »

0xfffafaCrash wrote: post #28
SoftTalker wrote: post #4

I think everything has a growing problem with LLM/AI generated content. Emails, blog posts, news articles, research papers, grant applications, business proposals, music, art, pretty much everything you can think of. There’s already more human produced content in the world than anyone could ever hope to consume, we don’t need more from AI.

> There’s already more human produced content in the world than anyone could ever hope to consume, we don’t need more from AI. Even if you think the harms of AI/machine generated content outweigh the good, this is not a winning argument. People don’t just consume arbitrary content for the sake of consuming any existing content. That’s rarely the point of it. People look for all kinds of things that don’t exist yet —…

The generated content is wasting the time of maintainers. How would you solve that?

For your winning argument, what would you use to prevent slop filling up your feed when there is more AI generated content, any sort of protocol that you have?

Re: LLM policy?

#90

Post by ixsploit »

dropbox_miner wrote: post #60
somenameforme wrote: post #12

Earlier quoted context omitted.

In some ways I'm starting to enjoy this. Do you remember the 419 scams? 'Hi, I'm a Nigerian prince named Michael Jordan. Give me $50 so I can buy some chemicals to clean a bunch of money I secretly stowed away and I'll send you $5000.' People actually used to fall for that. Of course some people probably still would (and a lot more certainly gets blocked by spam blockers) but I think overall society grew less substan…

I personally know people who look down upon people who use LLMs to write code. There is a lot of hate in some of senior developers that I talk to. I don't know if this growing tendency to be suspicious of AI usage is good or bad. For example, towards the final semester of my bachelors degree, my algorithms class started reporting students for academic misconduct because they the TAs started assuming that all the opti…

At the moment it is the other way around. LLMs rarely write good code if not instructed by someone that knows what they are doing. And even then the code is rarely good.
Post reply on HN