Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Mythos Finds a Curl Vulnerability

daniel.haxx.se

Post reply on HN
Print view View original

171–180 of 298 posts

Re: Mythos Finds a Curl Vulnerability

#171

Post by wnevets »

colechristensen wrote: post #165
wnevets wrote: post #148

Earlier quoted context omitted.

You've pointed codex to the entire source code of firefox and simply prompted it to find bugs and then had it write the exploits for you? Why haven't you published this? That would sink all of the the claude code hype.

No, I'm not interested in Firefox bugs, but I've done it with my own large projects. What I think happened here is an Anthropic team with very little security expertise were working on finding bugs for marketing reasons and when they prompted to make POC exploits of those bugs they didn't have much success because they didn't really know what to ask for. They then proceeded to very finely tune their next model to eag…

> No, I'm not interested in Firefox bugs, but I've done it with my own large projects.

Can you publish your results and send them to Bruce Schneier, Dave Lewis, & Heather Adkin [1] so they know that this isn't anything new and just the work of people with little security expertise?

[1] https://labs.cloudsecurityalliance.org/mythos-ciso/

Re: Mythos Finds a Curl Vulnerability

#172

Post by surgical_fire »

2ndorderthought wrote: post #145
surgical_fire wrote: post #123

Earlier quoted context omitted.

One phenomenon that spooks me is when intelligent people believe in idiotic things. It makes me wonder if there's a wrong turn in the road that I too might fall in the same pit.

As someone who watched one of their heros fall for some stupid cult like thing ten years ago and wondered the same thing. Then many years later fell for some dumb stuff. The answer is you probably will. Try to stay intellectually flexible, it'll be okay.

I am afraid of that, I wasn't joking.

I have seen people I consider as much smarter than me fall for some very idiotic things. I certainly don't consider myself immune.

I think that the advice to try being intellectually flexible is a good one. Strive to learn new things, expose yourself earnestly to ideas that challenge your beliefs, exercise empathy, etc

Re: Mythos Finds a Curl Vulnerability

#173

Post by orblivion »

dboreham wrote: post #170
orblivion wrote: post #168

Earlier quoted context omitted.

Is Mozilla marketing on Anthropic's behalf? As part of our continued collaboration with Anthropic, we had the opportunity to apply an early version of Claude Mythos Preview to Firefox. This week’s release of Firefox 150 includes fixes for 271 vulnerabilities identified during this initial evaluation. As these capabilities reach the hands of more defenders, many other teams are now experiencing the same vertigo we did…

I think it's more the cost to find a vulnerability that has significantly reduced, not the possibility that the vulnerability could have been found. But that cost mattered tremendously because someone has to fund the effort to find the bugs. This economics also applies to attackers.

Is Firefox less invested in this than Curl? I mean there must be some explanation for this.

Re: Mythos Finds a Curl Vulnerability

#174

Post by Pay08 »

orblivion wrote: post #168
rzmmm wrote: post #3

Quote: "My personal conclusion can however not end up with anything else than that the big hype around this model so far was primarily marketing. I see no evidence that this setup finds issues to any particular higher or more advanced degree than the other tools have done before Mythos. Maybe this model is a little bit better, but even if it is, it is not better to a degree that seems to make a significant dent in co…

Is Mozilla marketing on Anthropic's behalf? As part of our continued collaboration with Anthropic, we had the opportunity to apply an early version of Claude Mythos Preview to Firefox. This week’s release of Firefox 150 includes fixes for 271 vulnerabilities identified during this initial evaluation. As these capabilities reach the hands of more defenders, many other teams are now experiencing the same vertigo we did…

I certainly wouldn't be surprised if they were.

Re: Mythos Finds a Curl Vulnerability

#175

Post by spenczar5 »

orblivion wrote: post #168
rzmmm wrote: post #3

Quote: "My personal conclusion can however not end up with anything else than that the big hype around this model so far was primarily marketing. I see no evidence that this setup finds issues to any particular higher or more advanced degree than the other tools have done before Mythos. Maybe this model is a little bit better, but even if it is, it is not better to a degree that seems to make a significant dent in co…

Is Mozilla marketing on Anthropic's behalf? As part of our continued collaboration with Anthropic, we had the opportunity to apply an early version of Claude Mythos Preview to Firefox. This week’s release of Firefox 150 includes fixes for 271 vulnerabilities identified during this initial evaluation. As these capabilities reach the hands of more defenders, many other teams are now experiencing the same vertigo we did…

Yep! The industry term is "co-marketing" and its hard to avoid seeing once you spot it.

Re: Mythos Finds a Curl Vulnerability

#176

Post by spenczar5 »

orblivion wrote: post #173
dboreham wrote: post #170

Earlier quoted context omitted.

I think it's more the cost to find a vulnerability that has significantly reduced, not the possibility that the vulnerability could have been found. But that cost mattered tremendously because someone has to fund the effort to find the bugs. This economics also applies to attackers.

Is Firefox less invested in this than Curl? I mean there must be some explanation for this.

It's in the first sentence of your quote:

"our continued collaboration with Anthropic"

Read this as: "we get discounts, rate limit increases, a direct line to responsible product managers; in exchange we participate in friendly marketing." It's extremely common in this line of business - typical of database vendors, software tool companies, etc.

Re: Mythos Finds a Curl Vulnerability

#177

Post by orblivion »

spenczar5 wrote: post #176
orblivion wrote: post #173

Earlier quoted context omitted.

Is Firefox less invested in this than Curl? I mean there must be some explanation for this.

It's in the first sentence of your quote: "our continued collaboration with Anthropic" Read this as: "we get discounts, rate limit increases, a direct line to responsible product managers; in exchange we participate in friendly marketing." It's extremely common in this line of business - typical of database vendors, software tool companies, etc.

This is more in response to my original post, but okay interesting point. (When I said "invested" here I meant invested in finding security flaws.)

Re: Mythos Finds a Curl Vulnerability

#178

Post by orblivion »

spenczar5 wrote: post #175
orblivion wrote: post #168

Earlier quoted context omitted.

Is Mozilla marketing on Anthropic's behalf? As part of our continued collaboration with Anthropic, we had the opportunity to apply an early version of Claude Mythos Preview to Firefox. This week’s release of Firefox 150 includes fixes for 271 vulnerabilities identified during this initial evaluation. As these capabilities reach the hands of more defenders, many other teams are now experiencing the same vertigo we did…

Yep! The industry term is "co-marketing" and its hard to avoid seeing once you spot it.

I didn't think Mozilla was like that but duly noted.

Re: Mythos Finds a Curl Vulnerability

#179

Post by neuronexmachina »

hug wrote: post #119
embedding-shape wrote: post #115

Earlier quoted context omitted.

None of those other LLM tooling made the claims they're too dangerous to be released and used though, unlike Anthropic did with Mythos. What it highlights, is that Mythos doesn't seem so much better than other LLM driven tooling at finding security issues, which was the strongest claim Anthropic made in the first place.

Actually, OpenAI made a similar claim about one of their GPT models a while ago… Funnily enough that was while Dario Amodei was their research director.

If you're referring to gpt-2 in 2019, that primarily about concerns with it being used by spammers and fake content generators. In retrospect, that was a totally valid concern.

Re: Mythos Finds a Curl Vulnerability

#180

Post by neuronexmachina »

johnbarron wrote: post #59
therealpygon wrote: post #51

Earlier quoted context omitted.

Anthropic using marketing to convince people their models are more advanced, better built, or that AI is a threat that needs to be regulated because only they have the answer? I’m shocked. More seriously, so far I haven’t seen much indication that Mythos is more than Opus with a security focused code analysis harness. That said, the fact it can find these bugs in an automated fashion is the more important takeaway ou…

>> Anthropic using marketing to convince people their models are more advanced, better built, or that AI is a threat that needs to be regulated because only they have the answer? I’m shocked. I remember when OpenAI was saying GPT-2 was too dangerous to release.

Context from 2019: https://en.wikipedia.org/wiki/GPT-2

>While previous OpenAI models had been made immediately available to the public, OpenAI initially refused to make a public release of GPT-2's source code when announcing it in February, citing the risk of malicious use;[8][5] limited access to the model (i.e. an interface that allowed input and provided output, not the source code itself) was allowed for selected press outlets on announcement.[8] One commonly-cited justification was that, since generated text was usually completely novel, it could be used by spammers to evade automated filters; OpenAI demonstrated a version of GPT-2 fine-tuned to "generate infinite positive – or negative – reviews of products".[8]

>Another justification was that GPT-2 could be used to generate text that was obscene or racist. Researchers such as Jeremy Howard warned of "the technology to totally fill Twitter, email, and the web up with reasonable-sounding, context-appropriate prose, which would drown out all other speech and be impossible to filter".[18] ...

Post reply on HN