Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Mythos Finds a Curl Vulnerability

daniel.haxx.se

Post reply on HN
Print view View original

141–150 of 298 posts

Re: Mythos Finds a Curl Vulnerability

#141

Post by empath75 »

GuB-42 wrote: post #74
thombles wrote: post #66

Earlier quoted context omitted.

Curl simply isn't a good data point. It's one of the most picked-over codebases in existence with extensive security testing practices. All the researchers using not-quite-Mythos models have had plenty of time to report bugs up to this point. Daniel may be right that Mythos hasn't been a game changer for curl but the preconditions are different for virtually any other codebase. Perhaps the real marketing here is his…

To me, it is a very good data point. Curl uses all sorts of tools, including AI tools to find bugs. These tools, according to the article found hundreds of bugs including a dozen CVE. Mythos found one vulnerability. It means the Mythos is just another tool, not the revolution it claims to be. It is common that when a new tool is introduced that a bunch of bugs are found, with diminishing returns. Mythos finding one v…

It's not, really. Curl is an extraordinarily high value target that has already been picked over by well funded security researchers and state-sponsored groups using state of the art tooling for decades. That is not the target for which Mythos is a threat.

The threat isn't high value targets, which already had sophisticated folks picking over the code base using state of the art tools and tests, it's medium to low value targets which can now be picked over by random hackers who barely know anything about security themselves at a cost of a few dollars.

Re: Mythos Finds a Curl Vulnerability

#142

Post by iso1631 »

perching_aix wrote: post #30
iso1631 wrote: post #28

Earlier quoted context omitted.

War and Peace is about 590,000 words. Tiny compared to the full Harry Potter collection (about 1 million words over the 7 books), but long for a single book.

They're referring to the typo in the title, "Piece" vs "Peace". I also thought they were contending the word count before noticing. Even remarked how I find this a weird metric, given that code is not prose [0], but then I deleted that once I picked up on what's going on. [0] comparing the output of `wc -w` with the word counts of books I'm reasonably sure will be super off edit: ran a calc, substituting out symbols…

ahh interesting thanks

I guess it's related to the phenomenon where you can read words relatively easily as long as the first and last letters are correct and the rest of the letters are there.

https://wire.insiderfinance.io/the-brains-power-to-read-jumb...

Re: Mythos Finds a Curl Vulnerability

#143

Post by jrflo »

I know that the Mythos hype is part marketing by anthropic, but isn't it possible that with a highly scrutinized codebase, there just aren't any notable security exploits in it's current state? The fact that it found nothing isn't necessarily an incrimination against it, especially when other tools had identified hundreds of exploits previously. Seems like it's been completely picked over (for now).

Re: Mythos Finds a Curl Vulnerability

#144

Post by mschuster91 »

countWSS wrote: post #127
NitpickLawyer wrote: post #121

What's going on in this thread? It's weird how prevalent the negativity towards mythos is, and I'm not sure if it's people throwing the baby out with the bathwater or something more tinfoil-adjacent coordinated campaign. I also noticed this on a thread a few days ago, before the mozilla post. There were dozens of comments saying basically "mythos is vaporware". I get the idea that they're using it for marketing. Of c…

Here and on reddit, AI debugging is viewed as some weird shallow pattern-matching that obviously fails to spot real stuff and overload the maintainers. Instead of getting to "spotless record" of zero flaws, the people start rationalizing that "X is not a real bug" and inventing justifications for their(obviously bad) code, which is critique they can't accept from AI, only through human debate that they can't close wi…

> Here and on reddit, AI debugging is viewed as some weird shallow pattern-matching that obviously fails to spot real stuff and overload the maintainers.

That's because that is what a lot of people did in the last years [1] to pad their resumes or to force developers to backport patches to older (but supported) kernel versions that wouldn't have gone in if they didn't have a CVE attached [2]. Maintainers have been legitimately swamped with low-quality spam for a very long time. Only recently, in the last few months, AI actually got "good enough", the problem is that maintainers still have to differentiate between AI slop by wannabes and by AI-assisted reports reviewed and refined by actual human professionals.

[1] https://www.zdnet.com/article/how-fake-security-reports-are-...

[2] https://opensourcewatch.beehiiv.com/p/linux-gets-cve-securit...

Re: Mythos Finds a Curl Vulnerability

#145

Post by 2ndorderthought »

surgical_fire wrote: post #123
paradox242 wrote: post #105

Earlier quoted context omitted.

Considering Richard Dawkins has recently succumbed to the same delusion it is a reminder that no matter how intelligent someone may otherwise be, we are all human and have certain tendencies and blind spots; anthropomorphizing non-entities being one of those.

One phenomenon that spooks me is when intelligent people believe in idiotic things. It makes me wonder if there's a wrong turn in the road that I too might fall in the same pit.

As someone who watched one of their heros fall for some stupid cult like thing ten years ago and wondered the same thing. Then many years later fell for some dumb stuff. The answer is you probably will. Try to stay intellectually flexible, it'll be okay.

Re: Mythos Finds a Curl Vulnerability

#146

Post by ofjcihen »

colechristensen wrote: post #137
wnevets wrote: post #135

Earlier quoted context omitted.

> Opus can find bugs on its own in large codebases just fine with minimal prompting. and then it write the exploits automatically for you?

Yes

I will never ever understand how people are amazed by this. Have they just not tried it and then just assume that because Anthropic says this is the first it must be true?

This was one of the first things I tried and it works great.

Re: Mythos Finds a Curl Vulnerability

#148

Post by wnevets »

colechristensen wrote: post #139
wnevets wrote: post #138

Earlier quoted context omitted.

Can you send me that link?

What link? I've done it myself.

You've pointed codex to the entire source code of firefox and simply prompted it to find bugs and then had it write the exploits for you? Why haven't you published this? That would sink all of the the claude code hype.

Re: Mythos Finds a Curl Vulnerability

#149

Post by pixl97 »

flohofwoe wrote: post #40
AntiUSAbah wrote: post #12

There is always marketing involved and people should be able to put marketing into perspective. Also curl in this regard is a open source project, relativly small but critical, well known and used everywhere. Besides image libraries, tools like curl or sudo, su, passwd, etc. would also be my first try. Mythos is still not known at all what it can do. What does it mean from cost and benchmark pov to have a 10 Trillion…

> Nonetheless, the fact that LLMs got significant better in finding this, better than humans, started to happen half a year ago? *rolls eyes* regular static analyzers also have been "better than humans" for decades, being better than a human at a specific mechanical task really doesn't mean much. The interesting new thing is the type of potential "fuzzy bugs" described in the article that LLMs are able to identify (a…

Static analyzers are balls. For every real bug they find you are dealing with with piles of false positives and negatives.

Now, I'm not saying you shouldn't use them. They do catch the low hanging fruit. It's that LLMs actually have a much better understanding of things like intent when looking at your code and general architecture configurations that can lead to problems.

As you say we've had static analyzers forever, hence why they aren't dropping out 50 new CVE's a day. LLMs are. There is a massive stack of software out there that is getting analyzed and exploited at a rate faster than it's getting patched. Adding to that things like NPMs exploited package of the day and popular github repository takeovers this year looks massively different from last year in quantity and quality of exploits alone.

Re: Mythos Finds a Curl Vulnerability

#150

Post by atonse »

GuB-42 wrote: post #74
thombles wrote: post #66

Earlier quoted context omitted.

Curl simply isn't a good data point. It's one of the most picked-over codebases in existence with extensive security testing practices. All the researchers using not-quite-Mythos models have had plenty of time to report bugs up to this point. Daniel may be right that Mythos hasn't been a game changer for curl but the preconditions are different for virtually any other codebase. Perhaps the real marketing here is his…

To me, it is a very good data point. Curl uses all sorts of tools, including AI tools to find bugs. These tools, according to the article found hundreds of bugs including a dozen CVE. Mythos found one vulnerability. It means the Mythos is just another tool, not the revolution it claims to be. It is common that when a new tool is introduced that a bunch of bugs are found, with diminishing returns. Mythos finding one v…

I had a totally different take. The fact that Mythos found only one vulnerability is testament to how solid curl is, not how bad Mythos is.

Look at the Firefox blog post where they found something like 400 (or more) findings.

I have no doubt Mythos is very good at this, but I also don't think it's something unattainable by other labs within the next few months, with focus.

Post reply on HN