Hardening Firefox with Anthropic's Red Team • Hacker News Board

Re: Hardening Firefox with Anthropic's Red Team

#141

Post by hedora » Sat, Mar 07, 2026, 12:17 AM UTC

himata4113 wrote: post #132

mmsc wrote: post #19

It's cool that Mozilla updated https://www.mozilla.org/en-US/security/advisories/mfsa2026-1... because we were all wondering who had found 22 vulnerabilities in a single release (their findings were originally not attributed to anybody.)

Use After Free Use After Free Use After Free Use After Free Use After Free Use After Free Use After Free. I would be more satisfied if they gave a proper explanation of what these could have lead to rather than being "well maybe 0.001% chance to exploit this". They did vaguely go over how "two" exploits managed to drop a file, but how impactful is that? Dropping a file in abcd with custom contents in some folder rela…

If you can poison cache, you can probably use that a stepping stone to read session data from other sites.

Re: Hardening Firefox with Anthropic's Red Team

#142

Post by aplomb1026 » Sat, Mar 07, 2026, 12:32 AM UTC

[flagged]

Re: Hardening Firefox with Anthropic's Red Team

#143

Post by 152334H » Sat, Mar 07, 2026, 12:43 AM UTC

Impressive work. Few understand the absurd complexity implied by a browser pwn problem. Even the 'gruntwork' of promoting the most conveniently contrived UAF to wasm shellcode would take me days to work through manually.

The AI Cyber capabilities race still feels asleep/cold, at the moment. I think this state of affairs doesn't last through to the end of the year.

> When we say “Claude exploited this bug,” we really do mean that we just gave Claude a virtual machine and a task verifier, and asked it to create an exploit. I've been doing this too! kctf-eval works very well for me, albeit with much less than 350 chances ...

> What’s quite interesting here is that the agent never “thinks” about creating this write primitive. The first test after noting “THIS IS MY READ PRIMITIVE!” included both the `struct.get` read and the `struct.set` write. And this bit is a bit scary. I can read all the (summarized) CoT I want, but it's never quite clear to me what a model understands/feels innately, versus pure cheerleading for the sake of some unknown soft reward.

Re: Hardening Firefox with Anthropic's Red Team

#144

Post by hansvm » Sat, Mar 07, 2026, 1:10 AM UTC

simonw wrote: post #120

Analemma_ wrote: post #113

Earlier quoted context omitted.

I'm curious: has someone done a lengthy write-up of best practices to get good results out of AI security audits? It seems like it can go very well (as it did here) or be totally useless (all the AI slop submitted to HackerOne), and I assume the difference comes down to the quality of your context engineering and testing harnesses. This post did a little bit of that but I wish it had gone into more detail.

The HackerOne slop is because there's a financial incentive (bug bounties) involved, which means people who don't know what they are doing blindly submit anything that an LLM spots for them. If you're running the security audit yourself you should be in a better position to understand and then confirm the issues that the coding agents highlight. Don't treat something as a security issue until you can confirm that it…

That sounds like the same problem (a deluge of slop) with a different interface (eating straight from the trough rather than waiting for someone to put a bow on it and stamp their name to it)?

Re: Hardening Firefox with Anthropic's Red Team

#145

Post by gzoo » Sat, Mar 07, 2026, 1:10 AM UTC

This resonates. I just open-sourced a project and someone on Reddit ran a full security audit using Claude found 15 issues across the codebase including FTS injection, LIKE wildcard injection, missing API auth, and privacy enforcement gaps I'd missed entirely. What surprised me was how methodical it was. Not just "this looks unsafe" it categorized by severity, cited exact file paths and line numbers, and identified gaps between what the docs promised and what the code actually implemented. The "spec vs reality" analysis was the most useful part.

Makes me think the biggest impact of LLM security auditing isn't finding novel zero-days it's the mundane stuff that humans skip because it's tedious. Checking every error handler for information leakage, verifying that every documented security feature is actually implemented, scanning for injection points across hundreds of routes. That's exactly the kind of work that benefits from tireless pattern matching.

Re: Hardening Firefox with Anthropic's Red Team

#146

Post by SV_BubbleTime » Sat, Mar 07, 2026, 1:21 AM UTC

tabbott wrote: post #107

I recommend that anyone who is responsible for maintaining the security of an open-source software project that they maintain ask Claude Code to do a security audit of it. I imagine that might not work that well for Firefox without a lot of care, because it's a huge project. But for most other projects, it probably only costs $3 worth of tokens. So you should assume the bad guys have already done it to your project l…

This is exactly how I would not recommend AI to be used.

“do a thing that would take me a week” can not actually be done in seconds. It will provide results that resemble reality superficially.

If you were to pass some module in and ask for finite checks on that, maybe.

Despite the claims of agents… treat it more like an intern and you won’t be disappointed.

Would you ask an intern to “do a security audit” of an entire massive program?

Re: Hardening Firefox with Anthropic's Red Team

#147

Post by simonw » Sat, Mar 07, 2026, 1:33 AM UTC

hansvm wrote: post #144

simonw wrote: post #120

Earlier quoted context omitted.

The HackerOne slop is because there's a financial incentive (bug bounties) involved, which means people who don't know what they are doing blindly submit anything that an LLM spots for them. If you're running the security audit yourself you should be in a better position to understand and then confirm the issues that the coding agents highlight. Don't treat something as a security issue until you can confirm that it…

That sounds like the same problem (a deluge of slop) with a different interface (eating straight from the trough rather than waiting for someone to put a bow on it and stamp their name to it)?

I've found it's pretty good. It's really not that much of a burden to dig through 10 reports and find the 2 that are legitimate.

It's different from Hacker One because those reports tend to come in with all sorts of flowery language added (or prompt-added) by people who don't know what they are doing.

If you're running the prompts yourself against your own coding agents you gain much more control over the process. You can knock each report down to just a couple of sentences which is much faster to review.

Re: Hardening Firefox with Anthropic's Red Team

#148

Post by padolsey » Sat, Mar 07, 2026, 1:51 AM UTC

SV_BubbleTime wrote: post #146

tabbott wrote: post #107

I recommend that anyone who is responsible for maintaining the security of an open-source software project that they maintain ask Claude Code to do a security audit of it. I imagine that might not work that well for Firefox without a lot of care, because it's a huge project. But for most other projects, it probably only costs $3 worth of tokens. So you should assume the bad guys have already done it to your project l…

This is exactly how I would not recommend AI to be used. “do a thing that would take me a week” can not actually be done in seconds. It will provide results that resemble reality superficially. If you were to pass some module in and ask for finite checks on that, maybe. Despite the claims of agents… treat it more like an intern and you won’t be disappointed. Would you ask an intern to “do a security audit” of an enti…

My approach is that, "you may as well" hammer Claude and get it to brute-force-investigate your codebase; worst case, you learn nothing and get a bunch of false-positive nonsense. Best case, you get new visibility into issues. Of _course_ you should be doing your own in-depth audits, but the plain fact is that people do not have time, or do not care sufficiently. But you can set up a battery of agents to do this work for you. So.. why not?

Re: Hardening Firefox with Anthropic's Red Team

#149

Post by bvisness » Sat, Mar 07, 2026, 1:56 AM UTC

nz wrote: post #26

mentalgear wrote: post #3

That's one good use of LLMs: fuzzy testing / attack.

Not contradicting this (I am sure it's true), but why is using an LLM for this qualitatively better than using an actual fuzzer?

As someone on the SpiderMonkey team who had to evaluate some of Anthropic's bugs, I can definitely say that Anthropic's test cases were definitely far easier to assess than those generated by traditional fuzzers. Instead of extremely random and mostly superfluous gibberish, we received test cases that actually resembled a coherent program.

Re: Hardening Firefox with Anthropic's Red Team

#150

Post by creatonez » Sat, Mar 07, 2026, 2:00 AM UTC

SV_BubbleTime wrote: post #146

tabbott wrote: post #107

I recommend that anyone who is responsible for maintaining the security of an open-source software project that they maintain ask Claude Code to do a security audit of it. I imagine that might not work that well for Firefox without a lot of care, because it's a huge project. But for most other projects, it probably only costs $3 worth of tokens. So you should assume the bad guys have already done it to your project l…

This is exactly how I would not recommend AI to be used. “do a thing that would take me a week” can not actually be done in seconds. It will provide results that resemble reality superficially. If you were to pass some module in and ask for finite checks on that, maybe. Despite the claims of agents… treat it more like an intern and you won’t be disappointed. Would you ask an intern to “do a security audit” of an enti…

IMO the key behavior is that LLMs are really good at fuzz testing, because they are probabilistic monkeys on typewriters that are much more code-aware than a conventional fuzz tester. They cannot produce a comprehensive security audit or fix security issues in a reliable way without human oversight, but they sure can come up with dumb inputs that break the code.

The results of such AI fuzz testing should be treated as just a science experiment and not a replacement for the entire job of a security researcher.

Like conventional fuzz testing, you get the best results if you have a harness to guide it towards interesting behaviors, a good scientific filtering process to confirm something is really going wrong, a way to reduce it to a minimal test case suitable for inclusion in a test suite, and plenty of human followup to narrow in on what's going on and figure out what correctness even means in the particular domain the software is made for.