Earlier quoted context omitted.
I am assuming that millions of dollars have already been spent trying to get LLMs to hallucinate less. Even if Problems feel soluble, they often aren't. You might have to invent an entirely new paradigm of text generation to solve the hallucination problem. Or it could be the Collatz Conjecture of LLMs, that it "feels" so possible, but you never really get there.
Nuclear fusion was always 30 years away (c)
AI Slop vs. OSS Security
21–30 of 124 posts
Re: AI Slop vs. OSS Security
#22> The model has no concept of truth—only of plausibility. This is such an important problem to solve, and it feels soluble. Perhaps a layer with heavily biased weights, trained on carefully curated definitional data. If we could train in a sense of truth - even a small one - many of the hallucinatory patterns disappear. Hats off to the curl maintainers. You are the xkcd jenga block at the base.
Re: AI Slop vs. OSS Security
#23I know that this poses new problems (some people can't afford to spend this money), but it would be better than just wasting people's time.
Re: AI Slop vs. OSS Security
#24I don't know why submitting a vulnerability on those platforms is still free. If reporters had to pay a little amount of money (let's say, 20-50$, or indexed to the maximum gain of a vulnerability in a given category) when submitting their report, maybe those would be of better quality. I know that this poses new problems (some people can't afford to spend this money), but it would be better than just wasting people'…
Re: AI Slop vs. OSS Security
#25It's good for the site collecting the fee, it's good for the projects being reported on and it doesn't negatively affect valid reports.
It does exactly what we want by disincentivizing bad reports, either AI generated or not.
Re: AI Slop vs. OSS Security
#26Ironically, even this piece is significantly AI-generated: - Primarily relies on a single piece of evidence from the curl project, and expands it into multiple paragraphs - "But here's the gut punch:", "You're not building ... You're addressing ...", "This is the fundamental problem:" and so many other instances of Linkedin-esque writing. - The listicle under "What Might Actually Work"
Between this and the flip side of AI-slop it's getting really frustrating out here online.
Re: AI Slop vs. OSS Security
#27Ironically, even this piece is significantly AI-generated: - Primarily relies on a single piece of evidence from the curl project, and expands it into multiple paragraphs - "But here's the gut punch:", "You're not building ... You're addressing ...", "This is the fundamental problem:" and so many other instances of Linkedin-esque writing. - The listicle under "What Might Actually Work"
Yesterday my wife burst into my office: "You used AI to generate that (podcast) episode summary, we don't sound like that!" In point of fact, I had not. After the security reporting issue, the next problem on the list is "trust in other people's writing".
This has additional layers to it as well. For example, I actively avoid using em dash or anything that resembles it right now. If I had no exposure to the drama around AI, I wouldn't even be thinking about this. I am constraining my writing simply to avoid the implication.
Re: AI Slop vs. OSS Security
#28The solution isn't to block aggressively or to allow everything, but to prioritize. Put accounts older than the AI boom at the top, and allow them to give "referrals", ie stake a part of their own credibility to boost another account on the priority ladder. Referral systems are very efficient at filtering noise.
Re: AI Slop vs. OSS Security
#29Earlier quoted context omitted.
Yesterday my wife burst into my office: "You used AI to generate that (podcast) episode summary, we don't sound like that!" In point of fact, I had not. After the security reporting issue, the next problem on the list is "trust in other people's writing".
I think one potential downside of using LLMs or exposing yourself to their generated content is that you may subconsciously adopt their quirks over time. Even if you aren't actively using AI for a particular task, prior exposure to their outputs could be biasing your thoughts. This has additional layers to it as well. For example, I actively avoid using em dash or anything that resembles it right now. If I had no exp…
You don't know whose style the LLM would pick for that particular prompt and project. You might end up with Carmack or maybe that buggy, test-failing piece of junk project on Github.
Re: AI Slop vs. OSS Security
#30How ironic, considering every time I've reported a complicated issue to a program on HackerOne, the triggers have completely rejected them because they do not understand the complicated codebase that they are triaging for.
Also the curl examples given in TFA completely ignore recent developments, where curl's maintainers welcomed and fixed literally hundred of AI-found bugs: https://www.theregister.com/2025/10/02/curl_project_swamped_...