Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Hardening Firefox with Anthropic's Red Team

anthropic.com

Post reply on HN
Print view View original

61–70 of 188 posts

Re: Hardening Firefox with Anthropic's Red Team

#61

Post by halJordan »

stuxf wrote: post #11

It's interesting that they counted these as security vulnerabilities (from the linked Anthropic article) > “Crude” is an important caveat here. The exploits Claude wrote only worked on our testing environment, which intentionally removed some of the security features found in modern browsers. This includes, most importantly, the sandbox, the purpose of which is to reduce the impact of these types of vulnerabilities.…

I don't think it's appropriate to neg these vulnerabilities because another part of the system works. There are plenty of sandbox escapes. No one says don't fix the sandbox because you'll never get to the point of interrogation with the sandbox. Same here. Don't discount bugs just because a sandbox exists.

Re: Hardening Firefox with Anthropic's Red Team

#62

Post by john_strinlai »

tptacek wrote: post #55
pjmlp wrote: post #5

Earlier quoted context omitted.

Indeed, without it looks like a fluffy marketing piece.

And now that you know that it isn't, do you feel differently about the logic you used to write this comment?

i am curious, what are you hoping to get out of this comment? will you feel better if they say yes? what is your plan if they say no?

Re: Hardening Firefox with Anthropic's Red Team

#63

Post by amelius »

Quarrel wrote: post #56
amelius wrote: post #53

Earlier quoted context omitted.

Sounds good. Did you also test on old source code, to see if it could find the vulnerabilities that were already discovered by humans?

I really like this as a suggestion, but getting opensource code that isn't in the LLMs training data is a challenge. Then, with each model having a different training epoch, you end up with no useful comparison, to decide if new models are improving the situation. I don't doubt they are, just not sure this is a way to show it.

Yes, but perhaps the impact of being trained on code on being able to find bugs in code is not so large. You could do a bunch of experiments to find out. And this would be interesting in itself.

Re: Hardening Firefox with Anthropic's Red Team

#64

Post by tptacek »

john_strinlai wrote: post #62
tptacek wrote: post #55

Earlier quoted context omitted.

And now that you know that it isn't, do you feel differently about the logic you used to write this comment?

i am curious, what are you hoping to get out of this comment? will you feel better if they say yes? what is your plan if they say no?

I genuinely want to understand how they arrived at the claim that this was a fluffy marketing piece. Like, if you said on a different thread, "the Linux kernel is probably mostly written in Pascal", I would really want to understand how it was you got to that idea.

Re: Hardening Firefox with Anthropic's Red Team

#66

Post by mceachen »

suddenlybananas wrote: post #52
est31 wrote: post #51

I suppose eventually we'll see something like Google's OSS-Fuzz for core open source projects, maybe replacing bug bounty programs a bit. Anthropic already hands out Claude access for free to OSS maintainers. LLMs made it harder to run bug bounty programs where anyone can submit stuff, and where a lot of people flooded them with seemingly well-written but ultimately wrong reports. On the other hand, the newest genera…

> Anthropic already hands out Claude access for free to OSS maintainers. Free for 6 months after which it auto-renews if I recall correctly.

No mention of auto renewal is made as far as I (and Claude) could determine.

Their OSS offer is first-hit-is-free.

Re: Hardening Firefox with Anthropic's Red Team

#68

Post by tclancy »

deafpolygon wrote: post #41
tclancy wrote: post #32

Earlier quoted context omitted.

Yon Claude has a lean and hungry look.

An LLM by any other name would hallucinate the same

Anyone still reading down here will appreciate this https://bsky.app/profile/simeonthefool.bsky.social/post/3kbk...

Re: Hardening Firefox with Anthropic's Red Team

#69

Post by ycombinete »

amelius wrote: post #53
mozdeco wrote: post #40

Earlier quoted context omitted.

[working for Mozilla] That's because there were none. All bugs came with verifiable testcases (crash tests) that crashed the browser or the JS shell. For the JS shell, similar to fuzzing, a small fraction of these bugs were bugs in the shell itself (i.e. testing only) - but according to our fuzzing guidelines, these are not false positives and they will also be fixed.

Sounds good. Did you also test on old source code, to see if it could find the vulnerabilities that were already discovered by humans?

Isn’t that this from the (Anthropic) article:

“Our first step was to use Claude to find previously identified CVEs in older versions of the Firefox codebase. We were surprised that Opus 4.6 could reproduce a high percentage of these historical CVEs”

https://www.anthropic.com/news/mozilla-firefox-security

Re: Hardening Firefox with Anthropic's Red Team

#70

Post by rcxdude »

amelius wrote: post #53
mozdeco wrote: post #40

Earlier quoted context omitted.

[working for Mozilla] That's because there were none. All bugs came with verifiable testcases (crash tests) that crashed the browser or the JS shell. For the JS shell, similar to fuzzing, a small fraction of these bugs were bugs in the shell itself (i.e. testing only) - but according to our fuzzing guidelines, these are not false positives and they will also be fixed.

Sounds good. Did you also test on old source code, to see if it could find the vulnerabilities that were already discovered by humans?

Anthropic mention that they did beforehand, and it was the good performance it had there that lead to them looking for new bugs (since they couln't be sure that it was just memorising the vulnerabilities that had already been published).
Post reply on HN