I thought Mozilla Foundation were protecting us from AI.
Turns out it's the other way around - AI is protecting the Mozilla Foundation from us.
Hardening Firefox with Anthropic's Red Team
111–120 of 188 posts
Re: Hardening Firefox with Anthropic's Red Team
#112I've had mixed results. I find that agents can be great for: 1. Producing new tests to increase coverage. Migrating you to property testing. Setting up fuzzing. Setting up more static analysis tooling. All of that would normally take "time" but now it's a background task. 2. They can find some vulnerabilities. They are "okay" at this, but if you are willing to burn tokens then it's fine. 3. They are absolutely wrong…
But you're not a member of Anthropic's Red Team, with access to a specialist version of Claude.
Re: Hardening Firefox with Anthropic's Red Team
#113I recommend that anyone who is responsible for maintaining the security of an open-source software project that they maintain ask Claude Code to do a security audit of it. I imagine that might not work that well for Firefox without a lot of care, because it's a huge project. But for most other projects, it probably only costs $3 worth of tokens. So you should assume the bad guys have already done it to your project l…
I'm curious: has someone done a lengthy write-up of best practices to get good results out of AI security audits? It seems like it can go very well (as it did here) or be totally useless (all the AI slop submitted to HackerOne), and I assume the difference comes down to the quality of your context engineering and testing harnesses.
This post did a little bit of that but I wish it had gone into more detail.
Re: Hardening Firefox with Anthropic's Red Team
#114I recommend that anyone who is responsible for maintaining the security of an open-source software project that they maintain ask Claude Code to do a security audit of it. I imagine that might not work that well for Firefox without a lot of care, because it's a huge project. But for most other projects, it probably only costs $3 worth of tokens. So you should assume the bad guys have already done it to your project l…
I'm curious: has someone done a lengthy write-up of best practices to get good results out of AI security audits? It seems like it can go very well (as it did here) or be totally useless (all the AI slop submitted to HackerOne), and I assume the difference comes down to the quality of your context engineering and testing harnesses. This post did a little bit of that but I wish it had gone into more detail.
No mention of the quality of the engineers reviewing the result?
Re: Hardening Firefox with Anthropic's Red Team
#115I recommend that anyone who is responsible for maintaining the security of an open-source software project that they maintain ask Claude Code to do a security audit of it. I imagine that might not work that well for Firefox without a lot of care, because it's a huge project. But for most other projects, it probably only costs $3 worth of tokens. So you should assume the bad guys have already done it to your project l…
I'm curious: has someone done a lengthy write-up of best practices to get good results out of AI security audits? It seems like it can go very well (as it did here) or be totally useless (all the AI slop submitted to HackerOne), and I assume the difference comes down to the quality of your context engineering and testing harnesses. This post did a little bit of that but I wish it had gone into more detail.
We split our work:
* Specification extraction. We have security.md and policy.md, often per module. Threat model, mechanisms, etc. This is collaborative and gets checked in for ourselves and the AI. Policy is often tricky & malleable product/business/ux decision stuff, while security is technical layers more independent of that or broader threat model.
* Bug mining. It is driven by the above. It is iterative, where we keep running it to surface findings, adverserially analyze them, and prioritize them. We keep repeating until diminishing returns wrt priority levels. Likely leads to policy & security spec refinements. We use this pattern not just for security , but general bugs and other iterative quality & performance improvement flows - it's just a simple skill file with tweaks like parallel subagents to make it fast and reliable.
This lets the AI drive itself more easily and in ways you explicitly care about vs noise
Re: Hardening Firefox with Anthropic's Red Team
#116Missed a chance to take on Google by naming this effort Anthropic Project Zero
Re: Hardening Firefox with Anthropic's Red Team
#117It's interesting that they counted these as security vulnerabilities (from the linked Anthropic article) > “Crude” is an important caveat here. The exploits Claude wrote only worked on our testing environment, which intentionally removed some of the security features found in modern browsers. This includes, most importantly, the sandbox, the purpose of which is to reduce the impact of these types of vulnerabilities.…
[Work at Anthropic, used to work at Mozilla.] Firefox has never required a full chain exploit in order to consider something a vulnerability. A large proportion of disclosed Firefox vulnerabilities are vulnerabilities in the sandboxed process. If you look at Firefox's Security Severity Rating doc: https://wiki.mozilla.org/Security_Severity_Ratings/Client what you'll see is that vulnerabilities within the sandbox, and…
Semi-on topic. When will Anthropic make decisions on Claude Max for OSS maintainers? I would like to run this on my projects and some of my high-profile dependencies, but there was no update on the application.
Re: Hardening Firefox with Anthropic's Red Team
#118Re: Hardening Firefox with Anthropic's Red Team
#119> Firefox was not selected at random. It was chosen because it is a widely deployed and deeply scrutinized open source project — an ideal proving ground for a new class of defensive tools. What I was thinking was, "Chromium team is definitely not going to collaborate with us because they have Gemini, while Safari belongs to a company that operates in a notoriously secretive way when it comes to product development."
I would have started with Firefox, too. It is every bit as complex at Chromium, but as a project it has far fewer resources.
Re: Hardening Firefox with Anthropic's Red Team
#120I recommend that anyone who is responsible for maintaining the security of an open-source software project that they maintain ask Claude Code to do a security audit of it. I imagine that might not work that well for Firefox without a lot of care, because it's a huge project. But for most other projects, it probably only costs $3 worth of tokens. So you should assume the bad guys have already done it to your project l…
I'm curious: has someone done a lengthy write-up of best practices to get good results out of AI security audits? It seems like it can go very well (as it did here) or be totally useless (all the AI slop submitted to HackerOne), and I assume the difference comes down to the quality of your context engineering and testing harnesses. This post did a little bit of that but I wish it had gone into more detail.
The HackerOne slop is because there's a financial incentive (bug bounties) involved, which means people who don't know what they are doing blindly submit anything that an LLM spots for them.
If you're running the security audit yourself you should be in a better position to understand and then confirm the issues that the coding agents highlight. Don't treat something as a security issue until you can confirm that it is indeed a vulnerability. Coding agents can help you put that together but shouldn't be treated as infallible oracles.