Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLM policy?

github.com

Post reply on HN
Print view View original

101–110 of 145 posts

Re: LLM policy?

#101

Post by kijin »

imiric wrote: post #96
kijin wrote: post #41

I have a simple rule: whenever I receive an issue or PR from an unfamiliar account, I skim the text and/or code and post 1-2 quick questions. If the submitter responds intelligently, then the issue warrants a more in-depth investigation. If not, the submitter clearly doesn't think that the issue is worth following up on, so it's not worth my time, either. The more code a PR contains, the more thorough knowledge of it…

That's a reasonable first step, but a human can just as well feed your questions to the LLM, and post the responses. So you really have to be vigilant throughout your interactions with the author, or, if the PR is unsalvagable, save yourself some time and effort and reject it from the start. > I'd rather talk with an intelligent LLM There's no such thing.

The goal is not to filter out LLMs. The goal is to find and fix legitimate issues with my open-source software. I don't care if a human is feeding my questions to an LLM or an undead dog as long as they produce a decent signal.

If the noise increases to an uncomfortable level, of course, I may have to change my strategies. The point is that humans produce noise, too, sometimes even more than LLMs do.

Re: LLM policy?

#102

Post by somenameforme »

komali2 wrote: post #59
autoexec wrote: post #36

Earlier quoted context omitted.

I'm inclined to disagree just because photoshop has had a measurable effect on the population being skeptical of photos which at one point were practically treated as the gold standard of evidence. It's still easy to find people who have fallen for photoshopped images, but it's also easy to find people expressing doubts and insisting they can "tell by the pixels". Sometimes even legitimate photos get accused of being…

The other side of this is that ai tools are being treat like magic to the point that people are denying well documented events happened at all, such as the shooting of Charlie Kirk - conspiracies abound! Also bizarrely a subsection of the population seems to be really into blatantly ai generated images, just hop onto Facebook and see for yourself. I wonder if it has something to do with whatever monkey brain things m…

People of a profound initial bias will, in general, believe anything that supports that bias, and reject anything that challenges it, in both cases without any real consideration or thought whatsoever. So I don't think examples of individuals being "misled" by e.g. AI generated images or video, to extremes, is entirely realistic. Rather they were already at those extremes and will just eat up anything that appeals to those extremes.

To take a less politically charged example, imagine there is fake content 'proving' that the Moon landing is faked. Is that going to meaningfully sway people who don't have a major opinion one way or the other? Probably not, certainly not in meaningful numbers. And in general I think the truth does come out on most things. And when people find they have been misled, particularly if it was somebody they thought they could trust, it tends to result in a major rubber-banding in the opposite direction.

Re: LLM policy?

#103

Post by imiric »

kijin wrote: post #101
imiric wrote: post #96

Earlier quoted context omitted.

That's a reasonable first step, but a human can just as well feed your questions to the LLM, and post the responses. So you really have to be vigilant throughout your interactions with the author, or, if the PR is unsalvagable, save yourself some time and effort and reject it from the start. > I'd rather talk with an intelligent LLM There's no such thing.

The goal is not to filter out LLMs. The goal is to find and fix legitimate issues with my open-source software. I don't care if a human is feeding my questions to an LLM or an undead dog as long as they produce a decent signal. If the noise increases to an uncomfortable level, of course, I may have to change my strategies. The point is that humans produce noise, too, sometimes even more than LLMs do.

> The point is that humans produce noise, too, sometimes even more than LLMs do.

Humans can produce noise, but humans using LLMs can produce orders of magnitude more of it.

But your stance is reasonable. If it improves the product, who cares who/what produced it. I personally find reviewing machine-generated code and the process of code review with a machine much more exhausting than interactions with humans, but you may feel otherwise.

Re: LLM policy?

#104

Post by cyphar »

alexchantavy wrote: post #54

As an open source maintainer, I don't have an issue with AI; I have an issue with low quality slop whether it comes from a machine or from a human. The responsibility then is for an open source project to not be shy on calling out low quality/low effort work, have good integ tests and linters, and have guidance like AGENTS.md files that tell coding robots how to be successful in the repo.

> I have an issue with low quality slop whether it comes from a machine or from a human.

It's very hard for a human to mask a low-quality PR as thought it were reasonable quality. It is incredibly easy for an LLM to do it (in fact that is precisely what they are trained to do).

In addition, when I see a low-quality PR, I try to mentor the submitter. That doesn't make sense to do for LLMs (the submitter might not even bother reading my comments).

> and have guidance like AGENTS.md files that tell coding robots how to be successful in the repo.

If none of the maintainers use such tools, why is it our responsibility to maintain documentation to appease them (in the hopes of reducing spam)? A policy limiting their use is much simpler to write and requires less maintenance.

Re: LLM policy?

#105

Post by imiric »

Gigablah wrote: post #35
jmyeet wrote: post #14

Some years ago I read the Neal Stephenson book Anathem . SPOILERS: it has a version of the Internet called the Reticulum and one thing I remember is that it was filled with garbage. True information subtly changed multiple times until it was garbage. And there were agents to see through the garbage. I imagined this to be a neverending arms race. Honestly, this is kind of where I see LLM generated content going where…

Humans are far more effective at producing garbage than LLMs will ever be :)

What a ridiculously false thing to say.

A human can ask an LLM to generate megabytes of garbage data in seconds. No human could ever reach that level of effectiveness.

Re: LLM policy?

#106

Post by cyphar »

brabel wrote: post #98
rester324 wrote: post #85

Earlier quoted context omitted.

I think OP wanted to have a proof that the submitter can demonstrate understanding of both the system and the code in the PR. So how would a guidance solve that? Also OP didn't want to ban them I believe

> I think OP wanted to have a proof that the submitter can demonstrate understanding of both the system and the code in the PR. LLMs are really good at writing these. IF they think this will prove the author is human, they're mistaken.

> LLMs are really good at writing these. IF they think this will prove the author is human, they're mistaken.

That is not my general experience. LLM explanations of code tend to add extra specifics that are incorrect, and the whole thing looks like LLM output (lots of short sentences, overly cheery, too many dumb lists, and being overly repetitive).

Or course, a human could read the LLM output and then synthesise it in their own words, but they could also just read the code. I doubt someone will be able to convincingly act like they know what LLM code does just by consulting an LLM.

If your patch took you no time or effort to write and you took no interest in what it does, why should I (as a maintainer) bother looking at it and maintain it for the next 10 years? In any other circumstances we would rightfully call this spam and see it as a socially hostile activity.

Re: LLM policy?

#107

Post by darkwater »

kijin wrote: post #101
imiric wrote: post #96

Earlier quoted context omitted.

That's a reasonable first step, but a human can just as well feed your questions to the LLM, and post the responses. So you really have to be vigilant throughout your interactions with the author, or, if the PR is unsalvagable, save yourself some time and effort and reject it from the start. > I'd rather talk with an intelligent LLM There's no such thing.

The goal is not to filter out LLMs. The goal is to find and fix legitimate issues with my open-source software. I don't care if a human is feeding my questions to an LLM or an undead dog as long as they produce a decent signal. If the noise increases to an uncomfortable level, of course, I may have to change my strategies. The point is that humans produce noise, too, sometimes even more than LLMs do.

> The point is that humans produce noise, too, sometimes even more than LLMs do.

The core issue here, and it's something I'm seeing at work as well with "less talented" colleagues is that the kind of contributor that already produced noise passes the minimal threshold to use LLMs. But this happens without them understanding anything meaningful about how the software they are contributing to and if what the LLM generated makes sense or not. So, this makes them a 0.1x engineer with a 100x multiplier (for quantity, not quality).

Re: LLM policy?

#108

Post by theshrike79 »

vineyardmike wrote: post #6

I've seen an uptick in LLM generated bug reports from coworkers . A employee of my company (but not someone I work with regularly) used one of the CLI LLMs to search through logs for errors, and then automatically cut (hundreds!) of bugs to (sometimes) the correct teams. Turns out it was the result of some manager's mandate to "try integrating AI into our workflow". The resulting email was probably the least professi…

That's just stupid.

IF you want LLM code reviews, there are bots for codex, copilot, claude etc that plug straight into Github PRs and review the code automatically.

Some of them are actually useful, some just plain wrong and some are subtly wrong and you need to spend some time figuring out whether it's right or wrong :)

IMO it's still a net positive because LLMs tend to pick up really weird subtle errors that humans easily gloss over.

Re: LLM policy?

#109

Post by tsimionescu »

noduerme wrote: post #57
somenameforme wrote: post #12

Earlier quoted context omitted.

In some ways I'm starting to enjoy this. Do you remember the 419 scams? 'Hi, I'm a Nigerian prince named Michael Jordan. Give me $50 so I can buy some chemicals to clean a bunch of money I secretly stowed away and I'll send you $5000.' People actually used to fall for that. Of course some people probably still would (and a lot more certainly gets blocked by spam blockers) but I think overall society grew less substan…

Let's think about this algorithmically. If something "floods the zone with shit," it needs S amount of shit to cause a flood. But too much will eventually make the scam ineffectual. Widespread public distrust for the scam is (S+X)/time where X is the extra amount of shit beyond the minimum needed. Time is a global variable constrained by the rate at which people get burned or otherwise catch on to all other scams of…

> When it's all writing, all art, all music and all commentary on those things, it seems catastrophic. The whole cave is flooded with shit.

But writing in itself has been obviously untrustworthy since it started existing - something being written down doesn't in any way make it trustworthy. The fact that audio recording, photography, and video enjoyed this undeserved reputation of being inherently trustworthy was an accident of technology, and has come to an end.

Just like with writing, though, this doesn't signal a real problem of any kind. You should still only trust writing, audio, or video based on the source - as you always should have. All that's ending is the era of putting undue trust in audio/video from untrusted sources.

Of course, the big problems will be in the transition period, when most people still think they can trust these sources, or will think they can't trust actually trustable sources instead. But this will be temporary as things readjust.

And again, audio and video have been untrustworthy for a long time, for sensitive things. You should not have trusted video in itself even in the 40s - 50s, and audio and photos probably even in the 1910s were already somewhat easily manipulated. And this is even true in a legal context - audio or video evidence is not evidence in itself, it is only part of a witness testimony who can attest to the provenance and veracity.

Re: LLM policy?

#110

Post by watwut »

theshrike79 wrote: post #91
troupo wrote: post #67

Earlier quoted context omitted.

> Since they removed the "20+ minutes" button it has become basically useless to search on YT. A few days ago I definitely got into an A/B test where the search results were: - 5 shorts one under another - new section with one or two videos and one or two shorts - new section with five or more shorts in a horizontal layout - new section with videos of which 20-30% were shorts It's insane

They're pushing Shorts so fucking hard it makes me sick. Even worse is that we're banning TikTok because it's bad for the kids (short form algorithmic content), Snapchat (similar thing + strangers creeping) and Instagram Stories (algorithm again). BUT there is NO WAY for a parent to allow their kid to use Youtube AND block Shorts. (yes there are browser plugins etc, but how do you enforce them on a child?) And from w…

They are banning TikTok because it is not an American company and because it is profitable. They do not give a flying fck about kids.
Post reply on HN