Hardening Firefox with Anthropic's Red Team • Hacker News Board

Re: Hardening Firefox with Anthropic's Red Team

#121

Post by applfanboysbgon » Fri, Mar 06, 2026, 9:42 PM UTC

semiquaver wrote: post #65

It’s just a stochastic parrot! Somehow all these vulnerabilities were in the training data! Nothing ever happens! (/s if it’s not clear)

What an irritating comment. Identifying bugs in code is, in fact, exactly something a stochastic parrot could do. Vulnerability research is already a massively automated industry, and there's even a very well-established term -- "script kiddies" -- for malicious teenagers who run scripts that automatically find vulnerabilities in existing services without any knowledge of how they work. Having a new form of automation can certainly be a useful tool, but is still in no way an indication of "intelligence" or any deviation from the expected programming of next token prediction guided by statistical probability.

Re: Hardening Firefox with Anthropic's Red Team

#122

Post by nottorp » Fri, Mar 06, 2026, 9:44 PM UTC

halJordan wrote: post #61

stuxf wrote: post #11

It's interesting that they counted these as security vulnerabilities (from the linked Anthropic article) > “Crude” is an important caveat here. The exploits Claude wrote only worked on our testing environment, which intentionally removed some of the security features found in modern browsers. This includes, most importantly, the sandbox, the purpose of which is to reduce the impact of these types of vulnerabilities.…

I don't think it's appropriate to neg these vulnerabilities because another part of the system works. There are plenty of sandbox escapes. No one says don't fix the sandbox because you'll never get to the point of interrogation with the sandbox. Same here. Don't discount bugs just because a sandbox exists.

But doesn't this come from the company that said they had the "AI" write a compiler that can compile "linux" but couldn't compile a hello world in reality?

Re: Hardening Firefox with Anthropic's Red Team

#123

Post by hrmtst93837 » Fri, Mar 06, 2026, 9:51 PM UTC

nz wrote: post #26

mentalgear wrote: post #3

That's one good use of LLMs: fuzzy testing / attack.

Not contradicting this (I am sure it's true), but why is using an LLM for this qualitatively better than using an actual fuzzer?

[flagged]

Re: Hardening Firefox with Anthropic's Red Team

#124

Post by johannes1234321 » Fri, Mar 06, 2026, 10:29 PM UTC

simonw wrote: post #120

Analemma_ wrote: post #113

Earlier quoted context omitted.

I'm curious: has someone done a lengthy write-up of best practices to get good results out of AI security audits? It seems like it can go very well (as it did here) or be totally useless (all the AI slop submitted to HackerOne), and I assume the difference comes down to the quality of your context engineering and testing harnesses. This post did a little bit of that but I wish it had gone into more detail.

The HackerOne slop is because there's a financial incentive (bug bounties) involved, which means people who don't know what they are doing blindly submit anything that an LLM spots for them. If you're running the security audit yourself you should be in a better position to understand and then confirm the issues that the coding agents highlight. Don't treat something as a security issue until you can confirm that it…

The question still is: will enough useful stuff be included, to make it worth to dig through the slop? And how to tune the prompt to get better results.

Re: Hardening Firefox with Anthropic's Red Team

#125

Post by chill_ai_guy » Fri, Mar 06, 2026, 10:39 PM UTC

[flagged]

Re: Hardening Firefox with Anthropic's Red Team

#126

Post by simonw » Fri, Mar 06, 2026, 10:42 PM UTC

johannes1234321 wrote: post #124

simonw wrote: post #120

Earlier quoted context omitted.

The HackerOne slop is because there's a financial incentive (bug bounties) involved, which means people who don't know what they are doing blindly submit anything that an LLM spots for them. If you're running the security audit yourself you should be in a better position to understand and then confirm the issues that the coding agents highlight. Don't treat something as a security issue until you can confirm that it…

The question still is: will enough useful stuff be included, to make it worth to dig through the slop? And how to tune the prompt to get better results.

Best way to figure that out is to try it and see what happens.

Re: Hardening Firefox with Anthropic's Red Team

#127

Post by dmix » Fri, Mar 06, 2026, 10:46 PM UTC

mmsc wrote: post #19

It's cool that Mozilla updated https://www.mozilla.org/en-US/security/advisories/mfsa2026-1... because we were all wondering who had found 22 vulnerabilities in a single release (their findings were originally not attributed to anybody.)

Looks like a lot of the usual suspects

Re: Hardening Firefox with Anthropic's Red Team

#128

Post by bluGill » Fri, Mar 06, 2026, 10:46 PM UTC

johannes1234321 wrote: post #124

simonw wrote: post #120

Earlier quoted context omitted.

The HackerOne slop is because there's a financial incentive (bug bounties) involved, which means people who don't know what they are doing blindly submit anything that an LLM spots for them. If you're running the security audit yourself you should be in a better position to understand and then confirm the issues that the coding agents highlight. Don't treat something as a security issue until you can confirm that it…

The question still is: will enough useful stuff be included, to make it worth to dig through the slop? And how to tune the prompt to get better results.

That depends on how the tool is used. People who ask for a security vulnerability get slop. People who asked for deeper analysis often get something useful - but it isn't always a vulnerability.

Re: Hardening Firefox with Anthropic's Red Team

#129

Post by ronsor » Fri, Mar 06, 2026, 10:54 PM UTC

johannes1234321 wrote: post #124

simonw wrote: post #120

Earlier quoted context omitted.

The HackerOne slop is because there's a financial incentive (bug bounties) involved, which means people who don't know what they are doing blindly submit anything that an LLM spots for them. If you're running the security audit yourself you should be in a better position to understand and then confirm the issues that the coding agents highlight. Don't treat something as a security issue until you can confirm that it…

The question still is: will enough useful stuff be included, to make it worth to dig through the slop? And how to tune the prompt to get better results.

You're either digging through slop or digging through your whole codebase anyway.

Re: Hardening Firefox with Anthropic's Red Team

#130

Post by kwanbix » Fri, Mar 06, 2026, 10:58 PM UTC

mozdeco wrote: post #18

staticassertion wrote: post #17

I've had mixed results. I find that agents can be great for: 1. Producing new tests to increase coverage. Migrating you to property testing. Setting up fuzzing. Setting up more static analysis tooling. All of that would normally take "time" but now it's a background task. 2. They can find some vulnerabilities. They are "okay" at this, but if you are willing to burn tokens then it's fine. 3. They are absolutely wrong…

[work at Mozilla] I agree that LLMs are sometimes wrong, which is why this new method here is so valuable - it provides us with easily verifiable testcases rather than just some kind of analysis that could be right or wrong. Purely triaging through vulnerability reports that are static (i.e. no actual PoC) is very time consuming and false-positive prone (same issue with pure static analysis). I can't really confirm t…

Please, implement "name window" natively in Firefox.

I have to use chrome because the lack of it.