Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Mythos Finds a Curl Vulnerability

daniel.haxx.se

Post reply on HN
Print view View original

201–210 of 298 posts

Re: Mythos Finds a Curl Vulnerability

#201

Post by skywhopper »

atonse wrote: post #150
GuB-42 wrote: post #74

Earlier quoted context omitted.

To me, it is a very good data point. Curl uses all sorts of tools, including AI tools to find bugs. These tools, according to the article found hundreds of bugs including a dozen CVE. Mythos found one vulnerability. It means the Mythos is just another tool, not the revolution it claims to be. It is common that when a new tool is introduced that a bunch of bugs are found, with diminishing returns. Mythos finding one v…

I had a totally different take. The fact that Mythos found only one vulnerability is testament to how solid curl is, not how bad Mythos is. Look at the Firefox blog post where they found something like 400 (or more) findings. I have no doubt Mythos is very good at this, but I also don't think it's something unattainable by other labs within the next few months, with focus.

The point is that Anthropic claims it’s a huge leap over everything else. But it isn’t.

Re: Mythos Finds a Curl Vulnerability

#202

Post by EMM_386 »

If an AI agent finds zero bugs in a software utility, how can that be viewed in the sense the AI agent is not very good at finding bugs?

What if there are actually zero bugs?

> Five issues felt like nothing as we had expected an extensive list.

The expectation here may not match reality, but not necessarily because Mythos isn't as capable as claimed. curl may just happen to be a well-hardened tool that doesn't have too many security vulnerabilities in its present state.

Re: Mythos Finds a Curl Vulnerability

#203

Post by AtNightWeCode »

Should have scanned it with Mythos on an older code base before all these other sec issues was resolved with other tools. Or use the other tools to introduce the same kind of errors in other parts of the code base to see if Mythos would have found it.

A problem is that these tools seems smarter than they are cause they already read seen the answer key.

Re: Mythos Finds a Curl Vulnerability

#206

Post by u_fucking_dork »

NitpickLawyer wrote: post #121

What's going on in this thread? It's weird how prevalent the negativity towards mythos is, and I'm not sure if it's people throwing the baby out with the bathwater or something more tinfoil-adjacent coordinated campaign. I also noticed this on a thread a few days ago, before the mozilla post. There were dozens of comments saying basically "mythos is vaporware". I get the idea that they're using it for marketing. Of c…

> And then there's the team at mozilla

And then there’s the team at curl. Don’t fall for the cheap marketing stuff just because you like them

Everything points to Mythos being marginally better and nobody being able to afford to run it.

Re: Mythos Finds a Curl Vulnerability

#207

Post by rohit89 »

skywhopper wrote: post #201
atonse wrote: post #150

Earlier quoted context omitted.

I had a totally different take. The fact that Mythos found only one vulnerability is testament to how solid curl is, not how bad Mythos is. Look at the Firefox blog post where they found something like 400 (or more) findings. I have no doubt Mythos is very good at this, but I also don't think it's something unattainable by other labs within the next few months, with focus.

The point is that Anthropic claims it’s a huge leap over everything else. But it isn’t.

This depends on the actual number of undiscovered bugs still in curl. If there is nothing to find then even a 10x better Mythos will find nothing. Also I think the quality of the codebase matters a lot when it comes to finding bugs. Its possible that the curl is so well written that it is relatively straightforward for existing ai tools to find bugs.

Re: Mythos Finds a Curl Vulnerability

#208

Post by cmiles74 »

orblivion wrote: post #173
dboreham wrote: post #170

Earlier quoted context omitted.

I think it's more the cost to find a vulnerability that has significantly reduced, not the possibility that the vulnerability could have been found. But that cost mattered tremendously because someone has to fund the effort to find the bugs. This economics also applies to attackers.

Is Firefox less invested in this than Curl? I mean there must be some explanation for this.

I would expect Firefox to be less invested in this than Curl. Firefox is aimed at consumers, Curl is embedded in a wide variety of products.

Re: Mythos Finds a Curl Vulnerability

#209

Post by voxl »

JeremyNT wrote: post #103
rzmmm wrote: post #3

Quote: "My personal conclusion can however not end up with anything else than that the big hype around this model so far was primarily marketing. I see no evidence that this setup finds issues to any particular higher or more advanced degree than the other tools have done before Mythos. Maybe this model is a little bit better, but even if it is, it is not better to a degree that seems to make a significant dent in co…

This is roughly what I was assuming but of course the big caveat here is that they were already using the existing LLM driven tooling on an extensively audited codebase. So while anthropic's marketing may be hype there just wasn't much left to find, a point he makes in the blog post. Whether it's a big step forward for other kinds of projects is difficult to tell, but this highlights that everybody should be using AI…

Everyone should be using exclusively a proof assistant (Lean/Agda/Rocq/Isabelle) and proving their code correct, but they're not.

Do you see how ridiculous the zealotry sounds when its not your personal kind of zealotry?

Re: Mythos Finds a Curl Vulnerability

#210

Post by warkdarrior »

wongarsu wrote: post #198
HelloMcFly wrote: post #195

Earlier quoted context omitted.

I'll wear the dunce cap: how are you so certain this is co-marketing? I'm not saying you are wrong, but it doesn't seem obviously like marketing copy to me (which is of course what they'd want but that's nevertheless not in any way evidence one way or the other).

It starts with the words "As part of our continued collaboration with Anthropic" Once these words are used you can assume there is a contract stating how that collaboration works, and that this includes some sentences about how much each side is allowed to or required to say about it

So you claim that Mozilla entered into a contract with Anthropic, and said contract requires Mozilla to advertise for Anthropic on their blog. I hope Mozilla is getting a good payday out of this.
Post reply on HN