Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Mythos Finds a Curl Vulnerability

daniel.haxx.se

Post reply on HN
Print view View original

71–80 of 298 posts

Re: Mythos Finds a Curl Vulnerability

#71

Post by ontouchstart »

dotancohen wrote: post #13
bilekas wrote: post #4

> The single confirmed vulnerability is going to end up a severity low CVE planned to get published in sync with our pending next curl release 8.21.0 in late June My mind still cannot understand the quality and refinement that's gone into cURL. It really is the perfect example of something done so right, that people barely think twice about.

Curl and SQLite are my favourite examples of properly engineered, rigourously tested _anything_. It's really philosophical - those projects' contribution requirements demand such rigor, and the maintainers stand by that demand. A non-load-bearing document (not project code) is what makes that possible - very reminiscent of Einstein's thought experiments leading to tangible projects such as GPS or Descartes's belief t…

Some people must be working on training some models exclusively on high quality OSS code base like curl and SQLite without the noise of low quality training data.

I would do that with 100% local models from scratch.

Re: Mythos Finds a Curl Vulnerability

#72

Post by 20k »

thombles wrote: post #66
rzmmm wrote: post #3

Quote: "My personal conclusion can however not end up with anything else than that the big hype around this model so far was primarily marketing. I see no evidence that this setup finds issues to any particular higher or more advanced degree than the other tools have done before Mythos. Maybe this model is a little bit better, but even if it is, it is not better to a degree that seems to make a significant dent in co…

Curl simply isn't a good data point. It's one of the most picked-over codebases in existence with extensive security testing practices. All the researchers using not-quite-Mythos models have had plenty of time to report bugs up to this point. Daniel may be right that Mythos hasn't been a game changer for curl but the preconditions are different for virtually any other codebase. Perhaps the real marketing here is his…

We see this exact hypetrain every time a new model is released. Mythos simply hasn't lived up to the "we're all gunna die from the flood of vulnerabilities" hype even slightly. Its slightly better than previous models by all accounts, cool stuff

I've seen literally near word-for-word this exact chain of events multiple times previously

Re: Mythos Finds a Curl Vulnerability

#73

Post by 2ndorderthought »

johnbarron wrote: post #59
therealpygon wrote: post #51

Earlier quoted context omitted.

Anthropic using marketing to convince people their models are more advanced, better built, or that AI is a threat that needs to be regulated because only they have the answer? I’m shocked. More seriously, so far I haven’t seen much indication that Mythos is more than Opus with a security focused code analysis harness. That said, the fact it can find these bugs in an automated fashion is the more important takeaway ou…

>> Anthropic using marketing to convince people their models are more advanced, better built, or that AI is a threat that needs to be regulated because only they have the answer? I’m shocked. I remember when OpenAI was saying GPT-2 was too dangerous to release.

"it can almost like write 2 paragraphs!" "It might be conscious" "this is basically AGI, we had to fire someone who spilled the beans"

Re: Mythos Finds a Curl Vulnerability

#74

Post by GuB-42 »

thombles wrote: post #66
rzmmm wrote: post #3

Quote: "My personal conclusion can however not end up with anything else than that the big hype around this model so far was primarily marketing. I see no evidence that this setup finds issues to any particular higher or more advanced degree than the other tools have done before Mythos. Maybe this model is a little bit better, but even if it is, it is not better to a degree that seems to make a significant dent in co…

Curl simply isn't a good data point. It's one of the most picked-over codebases in existence with extensive security testing practices. All the researchers using not-quite-Mythos models have had plenty of time to report bugs up to this point. Daniel may be right that Mythos hasn't been a game changer for curl but the preconditions are different for virtually any other codebase. Perhaps the real marketing here is his…

To me, it is a very good data point.

Curl uses all sorts of tools, including AI tools to find bugs. These tools, according to the article found hundreds of bugs including a dozen CVE.

Mythos found one vulnerability. It means the Mythos is just another tool, not the revolution it claims to be.

It is common that when a new tool is introduced that a bunch of bugs are found, with diminishing returns. Mythos finding one vulnerability is consistent to what I would expect for a major update to an existing tool, which Mythos is over existing LLM-based solutions.

Re: Mythos Finds a Curl Vulnerability

#75

Post by OtherShrezzing »

perching_aix wrote: post #26

It's a shame he seems to reject the idea of actually diving in and using these tools interactively: > It’s not that I would have a lot of time to explore lots of different prompts and doing deep dive adventures anyway. His expertise I think would elevate the results quite a bit. Although if he never uses LLMs, which it reads like he doesn't, I guess it might backfire just as well. Prompting style (still?) does matter…

He posts about his use of language models a lot on Mastodon[0]. He does lots with language models, but doesn't buy all the way into the hype. I'd say he's one of. most reasonable & balanced voices on the subject of AI use in software today. Happy to use the technology, more than willing to push back on marketing bs.

[0] https://mastodon.social/@bagder

Re: Mythos Finds a Curl Vulnerability

#76

Post by stingraycharles »

johnbarron wrote: post #59
therealpygon wrote: post #51

Earlier quoted context omitted.

Anthropic using marketing to convince people their models are more advanced, better built, or that AI is a threat that needs to be regulated because only they have the answer? I’m shocked. More seriously, so far I haven’t seen much indication that Mythos is more than Opus with a security focused code analysis harness. That said, the fact it can find these bugs in an automated fashion is the more important takeaway ou…

>> Anthropic using marketing to convince people their models are more advanced, better built, or that AI is a threat that needs to be regulated because only they have the answer? I’m shocked. I remember when OpenAI was saying GPT-2 was too dangerous to release.

I remember when there was a guy at Google years a few years ago that was convinced that they had an internal, sentient creature in their labs (I think maybe 4 years ago?)

If I’m not mistaken, after the media cycle, he lost his job for breaking confidentiality.

That was the opposite of marketing, Google really didn’t get how to turn this into a product until ChatGPT happened.

Re: Mythos Finds a Curl Vulnerability

#77

Post by stingraycharles »

apexalpha wrote: post #67
fpesce wrote: post #65

Earlier quoted context omitted.

I don't agree with the "no tsunami in sight": if you don't look at 100+ bugs in Firefox and many more OSS projects, bunch of old unseen-before OpenBSD/Linux RCEs, and a few LPE in just 2 or 3 weeks for Linux itself... IMO, this does not sound like marketing scare, there is spike of vulnerability disclosures - high quality, low false positives - that can be sensed... It feels like we're speedrunning through few-years…

The LPEs were not found with Mythos but with existing, publicly available models.

And also: they did an earlier run with Opus to discover bugs (like segfaults).

In February, Opus discovered a whole bunch of security related bugs, but didn’t exploit them.

Mythos, in turn, was fed these bugs and told to exploit them.

Not saying it’s not impressive, but it was literally told “here are all the places our metal detector says there may be gold, please find gold”.

Re: Mythos Finds a Curl Vulnerability

#78

Post by thombles »

GuB-42 wrote: post #74
thombles wrote: post #66

Earlier quoted context omitted.

Curl simply isn't a good data point. It's one of the most picked-over codebases in existence with extensive security testing practices. All the researchers using not-quite-Mythos models have had plenty of time to report bugs up to this point. Daniel may be right that Mythos hasn't been a game changer for curl but the preconditions are different for virtually any other codebase. Perhaps the real marketing here is his…

To me, it is a very good data point. Curl uses all sorts of tools, including AI tools to find bugs. These tools, according to the article found hundreds of bugs including a dozen CVE. Mythos found one vulnerability. It means the Mythos is just another tool, not the revolution it claims to be. It is common that when a new tool is introduced that a bunch of bugs are found, with diminishing returns. Mythos finding one v…

The question is how many security vulnerabilities are actually left in the code after all the recent AI attention. Either Mythos is a nothingburger, or it's substantially more powerful but there's nothing left to do. Even a large amount of C can be correct eventually. Curl has the _potential_ to become a good data point maybe 6-12 months from now - if researchers and new tools find many more vulnerabilities then Mythos is proved to be hype. If they don't, then maybe Mythos is overkill for today's curl and its capabilities are better deployed elsewhere (like Firefox, apparently).

Re: Mythos Finds a Curl Vulnerability

#79

Post by spongebobstoes »

thombles wrote: post #66
rzmmm wrote: post #3

Quote: "My personal conclusion can however not end up with anything else than that the big hype around this model so far was primarily marketing. I see no evidence that this setup finds issues to any particular higher or more advanced degree than the other tools have done before Mythos. Maybe this model is a little bit better, but even if it is, it is not better to a degree that seems to make a significant dent in co…

Curl simply isn't a good data point. It's one of the most picked-over codebases in existence with extensive security testing practices. All the researchers using not-quite-Mythos models have had plenty of time to report bugs up to this point. Daniel may be right that Mythos hasn't been a game changer for curl but the preconditions are different for virtually any other codebase. Perhaps the real marketing here is his…

that makes it a good data point, because it is better able to illustrate the incremental capabilities of Mythos compared to previous tooling

that helps us to understand how much of Mythos is hype and how much is real

Re: Mythos Finds a Curl Vulnerability

#80

Post by Sharlin »

yjftsjthsd-h wrote: post #7

> Not particularly “dangerous” I'm not sure that follows. As noted, curl was already analyzed to death with every tool available; most software isn't at that level.

Curl is currently receiving a record number of high-quality bug/vuln reports (a rather sharp change from the earlier slop inundation), so it’s not like there’s nothing to find. Many or most of these are presumably found by human experts assisted by AI tools, but if Mythos were truly revolutionary, it should be able to find such issues on its own.

https://daniel.haxx.se/blog/2026/04/22/high-quality-chaos/, linked from TFA

Post reply on HN