Semi-related: I posted this yesterday: https://news.ycombinator.com/item?id=45828917
Different models perform differently when it comes to catching/fixing security vulnerabilities.
AI Slop vs. OSS Security
31–40 of 124 posts
Re: AI Slop vs. OSS Security
#32Just add a country IP ban, we all know who is submitting these reports. Remember Hacktoberfest?
Re: AI Slop vs. OSS Security
#33Very blunt maybe, but if individuals try to get internet points by doing frivolous security reports under their own name, should they be loudly pinned to a Wall of Shame to discourage the practice?
Re: AI Slop vs. OSS Security
#34Ironically, even this piece is significantly AI-generated: - Primarily relies on a single piece of evidence from the curl project, and expands it into multiple paragraphs - "But here's the gut punch:", "You're not building ... You're addressing ...", "This is the fundamental problem:" and so many other instances of Linkedin-esque writing. - The listicle under "What Might Actually Work"
Using the word is implies you have definite, conclusive proof, but the only one is a number of phrases that you believe are tells for AI generated stuff, but is it really or are you only now paying extra attention to it?
It's better to stay neutral and say you suspect it may be AI generated.
And for everyone else, responsible disclosure of using AI tools to write stuff would be appreciated.
(this comment did not involve AI. I don't know how to write an emdash)
Re: AI Slop vs. OSS Security
#35Ironically, even this piece is significantly AI-generated: - Primarily relies on a single piece of evidence from the curl project, and expands it into multiple paragraphs - "But here's the gut punch:", "You're not building ... You're addressing ...", "This is the fundamental problem:" and so many other instances of Linkedin-esque writing. - The listicle under "What Might Actually Work"
I'm so sick of people claiming things sound like AI, when it's so easily not true. Between this and the flip side of AI-slop it's getting really frustrating out here online.
I think people sometimes jump the gun over small things (emdashes, etc). That said, in this instance, your anger is very likely misdirected. The article is almost certainly substantially AI-generated.
Re: AI Slop vs. OSS Security
#36Just add a country IP ban, we all know who is submitting these reports. Remember Hacktoberfest?
That's a game of whack-a-mole, they'd just use a VPN. And besides, no we don't "all know" who is submitting these reports, that's a generalization.
Re: AI Slop vs. OSS Security
#37Earlier quoted context omitted.
Nuclear fusion was always 30 years away (c)
It would be nice if nuclear fusion had the AI budget.
Fusion will at best have a few dozen sales once it's commercially viable and then take decades to realise, but you can sell AI stuff to millions of customers for $20 / month each and do it today.
Re: AI Slop vs. OSS Security
#38Ironically, even this piece is significantly AI-generated: - Primarily relies on a single piece of evidence from the curl project, and expands it into multiple paragraphs - "But here's the gut punch:", "You're not building ... You're addressing ...", "This is the fundamental problem:" and so many other instances of Linkedin-esque writing. - The listicle under "What Might Actually Work"
My least favourite part of this timeline: anyone who writes well gets classified as AI. Some of us press Option+- to insert an em dash and have been for years.
Re: AI Slop vs. OSS Security
#39> This is the fundamental problem: AI can generate the form of security research without the substance. I think this is the fundamental problem of LLMs in general. Some of the time looks just enough right to seem legitimate. Luckily the rest of the time it doesn’t.
The other fundamental problem is that to a grifter, it's not a fundamental problem for the output to be plausible but often wrong. Plausible is all they need.
That's an important one. Another fundamental problem with plausible output is that it makes a manager, or a junior, or some other unsophisticated end user think the technology is almost there, and a reliably correct version is just around the corner.
Re: AI Slop vs. OSS Security
#40> The problem isn't knowledge—it's incentives.
> When you're volunteering out of love in a market society, you're setting yourself up to be exploited.
I sound like a broken record but there's unifying causes to most issues I observe in the world.
None of the proposed solutions address the cause (and they can't of course): public scrutiny doesn't do anything if account creation is zero-effort; monetary penalization will kill the submissions entirely.
In a perfect world OSS maintainers would get paid properly. But, we've been doing this since the 90s, and all that's happened is OSS got deployed by private companies, concentrating the wealth and the economic benefits. When every hour is paid labour, you pick the AWS Kafka over spinning up your own cluster, or you run Linux in the cloud instead of your own metal. This will always keep happening so long as the incentives are what they are and survival hinges on capital. That people still put in their free time speaks to the beautiful nature of humans, but it's in spite of the current systems.