Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Hardening Firefox with Anthropic's Red Team

anthropic.com

Post reply on HN
Print view View original

31–40 of 188 posts

Re: Hardening Firefox with Anthropic's Red Team

#31

Post by vorticalbox »

g947o wrote: post #30

> Firefox was not selected at random. It was chosen because it is a widely deployed and deeply scrutinized open source project — an ideal proving ground for a new class of defensive tools. What I was thinking was, "Chromium team is definitely not going to collaborate with us because they have Gemini, while Safari belongs to a company that operates in a notoriously secretive way when it comes to product development."

its just a different attack surface for safari they would need to blackbox attack the browser which is much harder than what they did her

Re: Hardening Firefox with Anthropic's Red Team

#33

Post by bell-cot »

kingkilr wrote: post #13
stuxf wrote: post #11

It's interesting that they counted these as security vulnerabilities (from the linked Anthropic article) > “Crude” is an important caveat here. The exploits Claude wrote only worked on our testing environment, which intentionally removed some of the security features found in modern browsers. This includes, most importantly, the sandbox, the purpose of which is to reduce the impact of these types of vulnerabilities.…

[Work at Anthropic, used to work at Mozilla.] Firefox has never required a full chain exploit in order to consider something a vulnerability. A large proportion of disclosed Firefox vulnerabilities are vulnerabilities in the sandboxed process. If you look at Firefox's Security Severity Rating doc: https://wiki.mozilla.org/Security_Severity_Ratings/Client what you'll see is that vulnerabilities within the sandbox, and…

If only this attitude was more common. All security is, ultimately, multi-ply Swiss cheese and unknown unknowns. In that environment, patching holes in your cheese layers is a critical part of statistical quality control.

Re: Hardening Firefox with Anthropic's Red Team

#34

Post by tclancy »

Part of that caught my eye. As yet another person who’s built a half-ass system of AI agents running overnight doing stuff, one thing I’ve tasked Claude with doing (in addition to writing tests, etc) is using formal verification when possible to verify solutions. It reads like that may be what Anthropic is doing in part.

And this is a good reminder for me to add a prompt about property testing being preferred over straight unit tests and maybe to create a prompt for fuzz testing the code when we hit Ready state.

Re: Hardening Firefox with Anthropic's Red Team

#35

Post by octoclaw »

staticassertion wrote: post #17

I've had mixed results. I find that agents can be great for: 1. Producing new tests to increase coverage. Migrating you to property testing. Setting up fuzzing. Setting up more static analysis tooling. All of that would normally take "time" but now it's a background task. 2. They can find some vulnerabilities. They are "okay" at this, but if you are willing to burn tokens then it's fine. 3. They are absolutely wrong…

[dead]

Re: Hardening Firefox with Anthropic's Red Team

#37

Post by rithdmc »

staticassertion wrote: post #17

I've had mixed results. I find that agents can be great for: 1. Producing new tests to increase coverage. Migrating you to property testing. Setting up fuzzing. Setting up more static analysis tooling. All of that would normally take "time" but now it's a background task. 2. They can find some vulnerabilities. They are "okay" at this, but if you are willing to burn tokens then it's fine. 3. They are absolutely wrong…

Security has had pattern matching in traditional static analysis for a while. It wasn't great.

I've personally used two AI-first static analysis security tools and found great results, including interesting business logic issues, across my employers SaaS tech stack. We integrated one of the tools. I look forward to getting employer approval to say which, but that hasn't happened yet, sadly.

Re: Hardening Firefox with Anthropic's Red Team

#39

Post by Analemma_ »

stuxf wrote: post #11

It's interesting that they counted these as security vulnerabilities (from the linked Anthropic article) > “Crude” is an important caveat here. The exploits Claude wrote only worked on our testing environment, which intentionally removed some of the security features found in modern browsers. This includes, most importantly, the sandbox, the purpose of which is to reduce the impact of these types of vulnerabilities.…

It's important to fix vulnerabilities even if they are blocked by the sandbox, because attackers stockpile partial 0-days in the hopes of using them in case a complementary exploit is found later. i.e. a sandbox escape doesn't help you on its own, but it's remotely possible someone was using one in combination with one of these fixed bugs and has now been thwarted. I consider this a straightforward success for security triage and fixing.

Re: Hardening Firefox with Anthropic's Red Team

#40

Post by mozdeco »

amelius wrote: post #38

Perhaps I missed it but I don't see any false positives mentioned.

[working for Mozilla]

That's because there were none. All bugs came with verifiable testcases (crash tests) that crashed the browser or the JS shell.

For the JS shell, similar to fuzzing, a small fraction of these bugs were bugs in the shell itself (i.e. testing only) - but according to our fuzzing guidelines, these are not false positives and they will also be fixed.

Post reply on HN