It’s just a stochastic parrot! Somehow all these vulnerabilities were in the training data! Nothing ever happens! (/s if it’s not clear)
What an irritating comment. Identifying bugs in code is, in fact, exactly something a stochastic parrot could do. Vulnerability research is already a massively automated industry, and there's even a very well-established term -- "script kiddies" -- for malicious teenagers who run scripts that automatically find vulnerabilities in existing services without any knowledge of how they work. Having a new form of automatio…
Thank you very much for acting as a useful foil and proving my point.
It's cool that Mozilla updated https://www.mozilla.org/en-US/security/advisories/mfsa2026-1... because we were all wondering who had found 22 vulnerabilities in a single release (their findings were originally not attributed to anybody.)
Use After Free Use After Free Use After Free Use After Free Use After Free Use After Free Use After Free.
I would be more satisfied if they gave a proper explanation of what these could have lead to rather than being "well maybe 0.001% chance to exploit this". They did vaguely go over how "two" exploits managed to drop a file, but how impactful is that? Dropping a file in abcd with custom contents in some folder relative to the user profile is not that impactful other than corrupting data or poisoning cache, injecting some javascript. Now reading session data from other sites, that I would find interesting.
The question still is: will enough useful stuff be included, to make it worth to dig through the slop? And how to tune the prompt to get better results.
Best way to figure that out is to try it and see what happens.
[claimed common problem exists, try X to find it] -> [Q about how to best do that] -> "the best way to do it is to do it yourself"
Surely people have found patterns that work reasonably well, and it's not "everyone is completely on their own"? I get that the scene is changing fast, but that's ridiculous.
Best way to figure that out is to try it and see what happens.
[claimed common problem exists, try X to find it] -> [Q about how to best do that] -> "the best way to do it is to do it yourself" Surely people have found patterns that work reasonably well, and it's not "everyone is completely on their own"? I get that the scene is changing fast, but that's ridiculous.
/security-review really is pretty good.
But your codebase is unique. Slop in one codebase is very dangerous in another.
I recommend that anyone who is responsible for maintaining the security of an open-source software project that they maintain ask Claude Code to do a security audit of it. I imagine that might not work that well for Firefox without a lot of care, because it's a huge project. But for most other projects, it probably only costs $3 worth of tokens. So you should assume the bad guys have already done it to your project l…
I'm curious: has someone done a lengthy write-up of best practices to get good results out of AI security audits? It seems like it can go very well (as it did here) or be totally useless (all the AI slop submitted to HackerOne), and I assume the difference comes down to the quality of your context engineering and testing harnesses. This post did a little bit of that but I wish it had gone into more detail.
The HackerOne slop is because there's a financial incentive (bug bounties) involved, which means people who don't know what they are doing blindly submit anything that an LLM spots for them. If you're running the security audit yourself you should be in a better position to understand and then confirm the issues that the coding agents highlight. Don't treat something as a security issue until you can confirm that it…
The question still is: will enough useful stuff be included, to make it worth to dig through the slop? And how to tune the prompt to get better results.
I assume it's just like asking for help refactoring, just targeting specific kinds of errors.
I ran a small python script that I made some years ago through an LLM recently and it pointed out several areas where the code would likely throw an error if certain inputs were received. Not security, but flaws nonetheless.
Best way to figure that out is to try it and see what happens.
[claimed common problem exists, try X to find it] -> [Q about how to best do that] -> "the best way to do it is to do it yourself" Surely people have found patterns that work reasonably well, and it's not "everyone is completely on their own"? I get that the scene is changing fast, but that's ridiculous.
There's so much superstition and outdated information out there that "try it yourself" really is good advice.
You can do that in conjunction with trying things other people report, but you'll learn more quickly from your own experiments. It's not like prompting a coding agent is expensive or time consuming, for the most part.
What an irritating comment. Identifying bugs in code is, in fact, exactly something a stochastic parrot could do. Vulnerability research is already a massively automated industry, and there's even a very well-established term -- "script kiddies" -- for malicious teenagers who run scripts that automatically find vulnerabilities in existing services without any knowledge of how they work. Having a new form of automatio…
Thank you very much for acting as a useful foil and proving my point.
You didn't make a point, and still haven't. You screeched a bunch of buzzphrases sarcastically as if that were equivalent to making a point, which is about par for the course for the level of reasoning (ie. none) shown by people with the position you hold. You seem to take it for granted that just by asserting that LLMs aren't next-token-prediction-programs, that must be factually true, without making any kind of argument or reasoning for why that is the case. Of course, any attempt to reason at that position falls apart under trivial scrutiny, so it's no wonder you're averse to reasoning about it and settle for trite assertions.