Live data from Hacker News

New serious vulnerabilities spiked around release of Claude Mythos Preview

epoch.ai

51–60 of 82 posts

Re: New serious vulnerabilities spiked around release of Claude Mythos Preview

#51
post #4

How are these reports verified to be valid? If there are too many some could be hallucinations too.

The best case scenario for AI companies is, people receive those bug reports, look at the model that produced it and not even look at the details, just apply the fix mindlessly This gives Anthropic a staggering amount of power. Oh it came from Mythos? We will just lose time trying to analyze it, better apply the fix ASAP

Currently it looks like the opposite is happening haha, "oh it came from AI, let's discard it ASAP" is the trend in open source

Re: New serious vulnerabilities spiked around release of Claude Mythos Preview

#52
post #4

How are these reports verified to be valid? If there are too many some could be hallucinations too.

We (Project Glasswing users) follow a proof-of-concept approach. We create the exploit and verify that it behaves as the AI claims. Given our experience as security engineers (many of us with 10+ YoE) we don’t simply report every critical bug Mythos claims to have found. We verify each one carefully. At least, that’s what most of the high-visibility users in Project Glasswing are doing. There are bad apples everywher…

> Some of the harnesses I’ve tested are quite weak, which leads to poor results.

So in your opinion, what would be the best off the shelf options? And secondly, how much better you’d say a purpose built one is compared to a general purpose one with a good system prompt?

Re: New serious vulnerabilities spiked around release of Claude Mythos Preview

#53

So basically there are two plausible explanations: 1. Someone with early access to Mythos leaked it to the bad guys. 2. Cybercriminals are getting enough mileage out of alternatives to Mythos to create exploits far more quickly, even though they don't have access to Mythos. My own guess is that it's a combination of #2 plus vibe-coding degrading software quality at multiple layers, open the door to sophisticated expl…

I think it's rather this:

3. People were already sitting on vulnerability reports from their own tools and threw them over the wall.

They were worried about getting scooped. They had to consider Mythos' alleged capabilities as a tool, and Project Glasswing potentially establishing a well-run disclosure and remediation process. Both could devalue preexisting results.

Re: New serious vulnerabilities spiked around release of Claude Mythos Preview

#54
One of the major differences between Amodei’s and Hagseth’s views is that Hagseth said that in their world they don’t distinguish between “defensive” and “offensive” capabilities.

In other words, a weapons missle defense system is equivalent to an attack one.

I think that applying this thinking to software is a mistake. A lot of commercial software uses open source libraries under the hood, and and while the large corporations might have access to Mythos/Fable/gpt 5.6, the open source library maintainers typically don’t. That leaves them vulnerable to foreign adversaries who do have access to AI models. Attackers don’t need Mythos-level capability then, they just need to outperform whatever the maintainers are using.

Which means that Anthropic’s decision to restrict security research on even Sonnet makes that gap (and thus an attackers opportunity) even larger.

I say this as a coder who wants to release some of my internal libraries to open source. The risk now is that I open up my own products (which use those libraries) to vulnerability scanners while not having those kinds of detection methods myself. This, it’s safer to not release and keep internal than to risk increasing my own attack risk.

Hopefully we will come to see that software is not equivalent to missle defense — writing safe code is different than attacking others’.

Re: New serious vulnerabilities spiked around release of Claude Mythos Preview

#55

Earlier quoted context omitted.

In pretty much every single HN post on this topic, there are a number of commenters claiming it’s false. Continued quantifiable data like this seems very important at hopefully resolving the ongoing disagreement about the facts.

I've seen plenty of people saying "Mythos isn't all that exceptional, lots of LLMs can find security vulnerabilities" -- and indeed there is some evidence for that; it sounds like Anthropic was taken somewhat by surprise at how easily a simple prompt managed to get Mythos to deliver exploits and didn't distinguish immediately between the effectiveness of Mythos and the effectiveness of the prompt. But the claim of "L…

I still have to see a single glibc bug that truly matters. I don't have illusions about our code quality, so there must be something to find.

We got many high-quality bug reports, some of them with a security aspect to them. Several of them received CVSSv3.1 scores of around 9.8 from the rating agencies, but these high numbers are misleading. (Vulnerability scoring is hard, and it's pretty much impossible for a library without reference to an application that uses the library.) Looking beyond the numbers, everything reported this year (and late in last year) was pretty harmless so far.

Does this mean LLMs are making a difference? For upstream developers, definitely. For end users? Not that much yet.

Maybe the picture changes once the organizations sitting on the good findings figure out how to disclose them to the relevant upstream projects. When I read the announcement of Project Glasswing, I immediately thought that this was going to be the hardest part.

Re: New serious vulnerabilities spiked around release of Claude Mythos Preview

#56

Earlier quoted context omitted.

> The best case scenario for AI companies is, people receive those bug reports, look at the model that produced it and not even look at the details, just apply the fix mindlessly Do people maintaining serious software do this, though?

The problem is that serious software is drowning in AI vulnerability reports. There is not enough manpower to analyze them properly. And if you ignore the reports (like curl is doing in their 1-month vacation), malicious actors will just exploit them. At some point it's inevitable to just rubber stamp whatever is coming from AI. The actual , underlying problem is that software is buggy and current programming languag…

We have plenty of functions that convert one byte array to another byte array. Both arrays have specified bounds. The functions are total (an error return indicates if the input or output arrays are incomplete). Most of them do not even have state that is preserved between calls. Complete source code is available in the same build for all the functions they call.

In theory, this should be very straightforward to prove correct with many of the current tools. In practice, no one has shown us how to do it. We could even rewrite the code from a macro/#include maze to proper function calls if that's a prerequisite for analysis. At this point, I would even take a one-off analysis.

Re: New serious vulnerabilities spiked around release of Claude Mythos Preview

#57

Earlier quoted context omitted.

I didn’t claim to have 10+ YoE; I said that most of the people in Project Glasswing are security researchers with 10+ YoE (avg). > Its very hard to understand what you're saying with the comment Yes, fair enough. I’m simply trying to shed some light on what goes on behind the scenes without disclosing too much information to avoid breaching the NDA(s) that all Project Glasswing users have signed. There’s a lot of spe…

[flagged]

I'm on HN to read comments. This is a social forum. Insight and opinions are the value proposition.

Re: New serious vulnerabilities spiked around release of Claude Mythos Preview

#58

One of the major differences between Amodei’s and Hagseth’s views is that Hagseth said that in their world they don’t distinguish between “defensive” and “offensive” capabilities. In other words, a weapons missle defense system is equivalent to an attack one. I think that applying this thinking to software is a mistake. A lot of commercial software uses open source libraries under the hood, and and while the large co…

If we take the noise about Mythos' capabilities as read, then releasing it freely into the world could result in chaos, as attackers find myriad new vulnerabilities and use them, and code owners frantically hunt for them and fix any that are exploited. (Noting, of course, how legendarily quick and agile large corporations aren't, compared to motivated individuals or small groups.). Eventually, given unfettered access to Mythos and sufficient time, things would settle down again once everything was patched, but who knows what would happen in the process?

So I suspect this has less to do with the underlying ethics or logic, and more to do with Anthropic not wanting to be held responsible for unleashing a potential period of chaos onto the world.

Of course, if someone has access to a tool that can find vulnerabilities in code, the process is identical whether the ultimate intent is to fix or exploit them (which may be Hegseth's underlying logic?). So to avoid this 'world chaos' scenario, Anthropic needs to somehow restrict Mythos access, avoiding bad players. And the only heuristics available at scale are either task-based assessment by AI (with downgrading of anything marginally risky to older models) or selection of trusted organisations by humans.

(By the by, to your point, it would also make sense to expand Glasswing to open source maintainers, at scale. I can't tell to what extent this has been part of that project?)

Re: New serious vulnerabilities spiked around release of Claude Mythos Preview

#59

Earlier quoted context omitted.

It's the former.

It's definitely both. Half the code my team puts into PR these days is dogshit.

Nah it's overwhelmingly the former, as far as what project Glasswing has focused on. It's finding vulnerabilities in code that was written years (in some cases decades) ago. Browsers, Linux Kernel, etc.

That's not to say that we aren't introducing new bugs, but I'm only addressing Mythos and Glasswing.

Post reply on HN