Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI Slop vs. OSS Security

devansh.bearblog.dev

Post reply on HN
Print view View original

91–100 of 124 posts

Re: AI Slop vs. OSS Security

#91

Post by dingnuts »

jerf wrote: post #84
imiric wrote: post #73

Earlier quoted context omitted.

Isn't the alternative far more likely? These tools were trained on the way people write in certain settings, which includes a lot of curated technical articles like this one, and we're seeing that echoed in their output. There's no "LLM style". There's "human style mimicked by LLMs". If they default to a specific style, then that's on the human user who chooses to go with it, or, likely, doesn't care. They could just…

There is no "LLM style". There is a "default LLM style", which is why I call it that. Or technically, one per LLM, but they seem to have converged pretty hard since they're all convergently evolving in the same environment. It's trivial to prompt it out of that style. Word about how to do it and that you should do it has gotten around in the academic world where the incentives to not be caught are high. So I don't ca…

the default LLM style is corporate voice lol

Re: AI Slop vs. OSS Security

#92

Post by dingnuts »

jobigoud wrote: post #74
thadt wrote: post #20

Earlier quoted context omitted.

Yesterday my wife burst into my office: "You used AI to generate that (podcast) episode summary, we don't sound like that!" In point of fact, I had not. After the security reporting issue, the next problem on the list is "trust in other people's writing".

Already a big problem in art, people go on witch hunt over what they think are signs of AI use. It's sad because people that are ok with AI art are still enjoying the human art just the same. Somehow their visceral hate of AI-art managed to ruin human art for themselves as well.

AI visual output mimicks art sufficiently that it is now more difficult to identify authenticity and humanity, which are important for the human connection audiences want from art.

AI outputs mimicking art rob audiences of the ability to appreciate art on its own in the wild without further markers of authenticity, which steals joy from a whole generation of digital artists that have grown up sharing their creativity with each other

If you lack the empathy to understand why AI art-like outputs are abhorrent, I hope someone wastes a significant portion of your near future with generated meaningless material presented to you as something that is valuable and was time consuming to make, and you gain nothing from it, so that you can understand the problem for yourself first hand.

Re: AI Slop vs. OSS Security

#93

Post by HiPhish »

We need a name for the type of attack where maintainers get overwhelmed by a massive amount of low-effort and false bug reports and contributions. I propose "Slop Overflow". That's right, you heard it here first, I came up with it :)

Re: AI Slop vs. OSS Security

#94

Post by DrewADesign »

> By flooding bug bounty programs with AI-generated reports, they feel productive and entrepreneurial. Some genuinely believe the AI has found something real. Others know it's questionable but figure they'll let the maintainers sort it out.

I’m not saying that AI hasn’t already given us useful things, but this is a symptom of one very negative change that’s drowned a lot of the positive out for many people: the competence gap used to be an automatic barrier for many things. For example, to get hired as a freelance developer, you had to have at least cargo-culted something together once or twice, and even if you were way overconfident in your capability, you probably knew you weren’t the real thing. However, the AI tools industry essentially markets free competence, and out of context for any given topic, that little disclaimer is meaningless. It’s essentially given people climbing the Dunning-Krueger Mt. Stupid the agency to produce garbage at a damaging volume that’s too plausible looking for them (or laypeople, for that matter) to realize it’s garbage. I also think somewhat nihilist people prone to get-rich-quick schemes (e.g. drop-shipping, NFTs) play these workflows like lottery tickets while remaining deliberately ignorant of their dubious value.

Re: AI Slop vs. OSS Security

#95

Post by [deleted] »

supriyo-biswas wrote: post #19

Ironically, even this piece is significantly AI-generated: - Primarily relies on a single piece of evidence from the curl project, and expands it into multiple paragraphs - "But here's the gut punch:", "You're not building ... You're addressing ...", "This is the fundamental problem:" and so many other instances of Linkedin-esque writing. - The listicle under "What Might Actually Work"

[deleted]

Re: AI Slop vs. OSS Security

#97

Post by [deleted] »

supriyo-biswas wrote: post #19

Ironically, even this piece is significantly AI-generated: - Primarily relies on a single piece of evidence from the curl project, and expands it into multiple paragraphs - "But here's the gut punch:", "You're not building ... You're addressing ...", "This is the fundamental problem:" and so many other instances of Linkedin-esque writing. - The listicle under "What Might Actually Work"

[deleted]

Re: AI Slop vs. OSS Security

#98

Post by gdulli »

jimbokun wrote: post #80
Sharlin wrote: post #47

Earlier quoted context omitted.

Unfortunately, to a majority of the population approximately 100% of LLM output seems entirely legitimate.

Except for things they happen to know something about.

Unfortunately, too few people are making the obvious leap from "LLMs aren't great for topics I have expertise in" to "maybe that means LLMs aren't actually great for the other topics either."

Re: AI Slop vs. OSS Security

#99

Post by whywhywhywhy »

jobigoud wrote: post #74
thadt wrote: post #20

Earlier quoted context omitted.

Yesterday my wife burst into my office: "You used AI to generate that (podcast) episode summary, we don't sound like that!" In point of fact, I had not. After the security reporting issue, the next problem on the list is "trust in other people's writing".

Already a big problem in art, people go on witch hunt over what they think are signs of AI use. It's sad because people that are ok with AI art are still enjoying the human art just the same. Somehow their visceral hate of AI-art managed to ruin human art for themselves as well.

This ultimately will only ever harm human artists accused of it. AI artists can just say “yeah, I did, so what” defusing the criticism.

Re: AI Slop vs. OSS Security

#100

Post by Rebuff5007 »

Sharlin wrote: post #47
goalieca wrote: post #9

> This is the fundamental problem: AI can generate the form of security research without the substance. I think this is the fundamental problem of LLMs in general. Some of the time looks just enough right to seem legitimate. Luckily the rest of the time it doesn’t.

Unfortunately, to a majority of the population approximately 100% of LLM output seems entirely legitimate.

And a sizable portion of the population believe vaccines don't work and/or have 5G!

I feel like I'm watching a tsunami about to hit while literally already drowning from a different tsunami.

Post reply on HN