Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Mythos Finds a Curl Vulnerability

daniel.haxx.se

Post reply on HN
Print view View original

241–250 of 298 posts

Re: Mythos Finds a Curl Vulnerability

#241

Post by bilekas »

pjmlp wrote: post #238
bilekas wrote: post #234

Earlier quoted context omitted.

> Easy, it shows what is achievable if there is a high bar for quality in every single line of code that gets commited This is becoming a more and more overlooked/underrated feature. I genuinely believe it would be impossible in any company that depends on shareholder value. I am yet to convince any company I've worked in without bloody hands that we need to solve old tech debt and refactor certain things etc.

Which is liability is relevant, that is the only language shareholders understand.

If you can get that message across the right way, you're a better company man than me. There's always someone more important than me to say 'but this needs to be delivered first'.

Re: Mythos Finds a Curl Vulnerability

#242

Post by colechristensen »

wnevets wrote: post #192
colechristensen wrote: post #183

Earlier quoted context omitted.

That whitepaper did not need 19 authors. They're there for show. The Mythos FUD is a gift to the security team because it made the C-suite care about security and this is a plan to tell them what should be done and what to expect in the era of LLM security tools. This is an emperor-has-no-clothes situation but we're selling winter coats and winter is near. Not focusing on how the Mythos FUD is exaggeration and instea…

Isn't that all the more reason to publish your process & results using Codex to do the same thing they're claiming? Presuming any bugs Codex found would be fixed and no longer a security concern.

No, what I'm doing isn't remarkable.

Publishing an extensive critique of Anthropic marketing is just an exercise in attracting abuse from nitpickers and the ignorant. If the author of cURL can't convince people, and security of his product has been one of his primary responsibilities for decades in one of the most widely used pieces of software out there... what hope do I have?

I've got better things to do.

Re: Mythos Finds a Curl Vulnerability

#243

Post by danudey »

embedding-shape wrote: post #115
JeremyNT wrote: post #103

Earlier quoted context omitted.

This is roughly what I was assuming but of course the big caveat here is that they were already using the existing LLM driven tooling on an extensively audited codebase. So while anthropic's marketing may be hype there just wasn't much left to find, a point he makes in the blog post. Whether it's a big step forward for other kinds of projects is difficult to tell, but this highlights that everybody should be using AI…

None of those other LLM tooling made the claims they're too dangerous to be released and used though, unlike Anthropic did with Mythos. What it highlights, is that Mythos doesn't seem so much better than other LLM driven tooling at finding security issues, which was the strongest claim Anthropic made in the first place.

It's important to keep in mind that very, very few projects are as rigorously tested as curl, so while it's interesting to hear this feedback I think curl would be a torture test for any security scanning. I'd be more interested to hear about other random libraries that aren't as thoroughly analyzed as curl; show me some results for GnuTLS, for example, or dpkg/rpm/apt/dnf/pacman/etc.

Re: Mythos Finds a Curl Vulnerability

#244

Post by muwtyhg »

mistrial9 wrote: post #225
jabwd wrote: post #120

Earlier quoted context omitted.

They most likely understood that it wasn't viable for anything. OpenAI just yolo'd it and now we're dealing with the fallout. I'm fairly certain that any management layer at google isn't going to say yes to "invest 5 billion to make 10 million" scheme that OpenAI, Anthropic, are currently running.

"ChatGPT has over 900 million weekly active users worldwide. ... ChatGPT Plus has around 50 million paying subscribers"

What you have typed does not address anything the person you are responding to said.

With those 50 million subscribers, how much do they pay and how much do they cost? That is the only relevant piece of information when discussing the investment and returns of OpenAI.

Re: Mythos Finds a Curl Vulnerability

#245

Post by yjftsjthsd-h »

fpesce wrote: post #65
apexalpha wrote: post #38

> An amazingly successful marketing stunt for sure. This. Well done by Antropic. It even reached the CISO of my small semi-government org in the Netherlands, who slightly panicked at the announced 'tsunami' of vulnerabilities that was coming with Mythos. Got us some more money and priority with the board, though. Never waste a good marketing scare.

I don't agree with the "no tsunami in sight": if you don't look at 100+ bugs in Firefox and many more OSS projects, bunch of old unseen-before OpenBSD/Linux RCEs, and a few LPE in just 2 or 3 weeks for Linux itself... IMO, this does not sound like marketing scare, there is spike of vulnerability disclosures - high quality, low false positives - that can be sensed... It feels like we're speedrunning through few-years…

> bunch of old unseen-before OpenBSD/Linux RCEs,

AFAIK, the only thing it found in OpenBSD was a DoS?

Edit: For that matter, I'm not aware of RCEs in Linux, only LPE?

Re: Mythos Finds a Curl Vulnerability

#246

Post by AgentME »

iterateoften wrote: post #126
embedding-shape wrote: post #115

Earlier quoted context omitted.

None of those other LLM tooling made the claims they're too dangerous to be released and used though, unlike Anthropic did with Mythos. What it highlights, is that Mythos doesn't seem so much better than other LLM driven tooling at finding security issues, which was the strongest claim Anthropic made in the first place.

People love defending Anthropics shortcomings… “Mythos isn’t supposed to be that good at security, because actually Anthropic was referring more about running llms than mythos specifically” “The opus model is worse because they have no compute because they are training mythos. The degraded performance is justified!” “All the bugs in Claude code is just because the models are so good they are just looping and are ship…

It's silly to act like they've got mud on their face when Mythos and Opus are apparently some of the very best models. Anyone that has found value out of previous LLMs is likely to find more value out of the newest ones. The only thing Mythos looks bad against is the very tall bar some people have imagined. People are putting too much weight on marketing and then reaction to marketing.

Re: Mythos Finds a Curl Vulnerability

#248

Post by embedding-shape »

AgentME wrote: post #246
iterateoften wrote: post #126

Earlier quoted context omitted.

People love defending Anthropics shortcomings… “Mythos isn’t supposed to be that good at security, because actually Anthropic was referring more about running llms than mythos specifically” “The opus model is worse because they have no compute because they are training mythos. The degraded performance is justified!” “All the bugs in Claude code is just because the models are so good they are just looping and are ship…

It's silly to act like they've got mud on their face when Mythos and Opus are apparently some of the very best models. Anyone that has found value out of previous LLMs is likely to find more value out of the newest ones. The only thing Mythos looks bad against is the very tall bar some people have imagined. People are putting too much weight on marketing and then reaction to marketing.

> People are putting too much weight on marketing and then reaction to marketing.

No, what others are doing, which I've done myself in the past too, is to evaluate how much their marketing matches up with reality, then share our experience about that. Very different than just "putting too much weight on marketing".

Re: Mythos Finds a Curl Vulnerability

#249

Post by stirfish »

embedding-shape wrote: post #115
JeremyNT wrote: post #103

Earlier quoted context omitted.

This is roughly what I was assuming but of course the big caveat here is that they were already using the existing LLM driven tooling on an extensively audited codebase. So while anthropic's marketing may be hype there just wasn't much left to find, a point he makes in the blog post. Whether it's a big step forward for other kinds of projects is difficult to tell, but this highlights that everybody should be using AI…

None of those other LLM tooling made the claims they're too dangerous to be released and used though, unlike Anthropic did with Mythos. What it highlights, is that Mythos doesn't seem so much better than other LLM driven tooling at finding security issues, which was the strongest claim Anthropic made in the first place.

Too dangerous to be released, right after the Department of Defense* dropped them

Re: Mythos Finds a Curl Vulnerability

#250

Post by bamboozled »

AgentME wrote: post #240
helloplanets wrote: post #86

Earlier quoted context omitted.

Anthropic has is quickly destroying customer goodwill by repeatedly pulling the same stunt. Horrible marketing, imho. It's an entirely different thing to have the company conduct research on LLMs in general being a cybersecurity threat, instead of going " our new model is just too powerful" and shift the discussion to revolve around that. It's slimey.

Hasn't almost every new frontier model had an early period of limited access? I don't get why everyone is acting like Mythos is particularly egregious for this.

It is called "Mythos" dude...do you have any idea how mysterious and scary this sounds to most people and how much hype that alone can generate.

If the model was calle "Mini Mouse" it wouldn't feel anywhere near as threatening and interesting.

It sounds like the name of a cologne from the 70s or something and I like it.

Post reply on HN