Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Mythos Finds a Curl Vulnerability

daniel.haxx.se

Post reply on HN
Print view View original

111–120 of 298 posts

Re: Mythos Finds a Curl Vulnerability

#111

Post by cvwright »

rzmmm wrote: post #3

Quote: "My personal conclusion can however not end up with anything else than that the big hype around this model so far was primarily marketing. I see no evidence that this setup finds issues to any particular higher or more advanced degree than the other tools have done before Mythos. Maybe this model is a little bit better, but even if it is, it is not better to a degree that seems to make a significant dent in co…

Even that press release never claimed that Mythos was better than Opus at finding bugs.

They claim the huge advance is in exploiting the bugs.

Re: Mythos Finds a Curl Vulnerability

#112

Post by shakna »

vidarh wrote: post #20
rzmmm wrote: post #3

Quote: "My personal conclusion can however not end up with anything else than that the big hype around this model so far was primarily marketing. I see no evidence that this setup finds issues to any particular higher or more advanced degree than the other tools have done before Mythos. Maybe this model is a little bit better, but even if it is, it is not better to a degree that seems to make a significant dent in co…

It may well be that the hype was primarily marketing. The other alternative is that Curl is simply secure enough that there was far less to find than in other projects.

Daniel found 30 CVEs in Curl, this year. I would not say that there is nothing to find, here. Just that it takes an actual expert.

Re: Mythos Finds a Curl Vulnerability

#113

Post by perching_aix »

OtherShrezzing wrote: post #75
perching_aix wrote: post #26

It's a shame he seems to reject the idea of actually diving in and using these tools interactively: > It’s not that I would have a lot of time to explore lots of different prompts and doing deep dive adventures anyway. His expertise I think would elevate the results quite a bit. Although if he never uses LLMs, which it reads like he doesn't, I guess it might backfire just as well. Prompting style (still?) does matter…

He posts about his use of language models a lot on Mastodon[0]. He does lots with language models, but doesn't buy all the way into the hype. I'd say he's one of. most reasonable & balanced voices on the subject of AI use in software today. Happy to use the technology, more than willing to push back on marketing bs. [0] https://mastodon.social/@bagder

I see, thanks.

I checked back two weeks worth of posts, reposts, and replies there, and do not see anything suggesting so, so I'll have to take your word for this.

What I do see is him responding to seemingly rather frequent harassment about AI use @ curl however. The stance he takes in those cases is very reasonable (even if you don't use AI for scanning the codebase and contributions, threat actors will), it's unfortunate this topic is so political that he has to deal with this to such an extent.

Re: Mythos Finds a Curl Vulnerability

#114

Post by Aurornis »

fpesce wrote: post #65
apexalpha wrote: post #38

> An amazingly successful marketing stunt for sure. This. Well done by Antropic. It even reached the CISO of my small semi-government org in the Netherlands, who slightly panicked at the announced 'tsunami' of vulnerabilities that was coming with Mythos. Got us some more money and priority with the board, though. Never waste a good marketing scare.

I don't agree with the "no tsunami in sight": if you don't look at 100+ bugs in Firefox and many more OSS projects, bunch of old unseen-before OpenBSD/Linux RCEs, and a few LPE in just 2 or 3 weeks for Linux itself... IMO, this does not sound like marketing scare, there is spike of vulnerability disclosures - high quality, low false positives - that can be sensed... It feels like we're speedrunning through few-years…

Mythos isn’t released yet.

Anthropic noticed the trend of AI vulnerability scanning and started advertising Mythos, which is unreleased, as being very good at it.

Then they donated very large token budgets for using Mythos privately to several teams. Those teams used the free token spend for security research (that was the deal) and anything they found got attributed to Mythos, not the token budget.

Mythos looks like a good incremental model but the PR team has done a great job of associating themselves with the current trend. So much so that comments like yours already associated vulnerabilities found with this model which isn’t even available yet

Re: Mythos Finds a Curl Vulnerability

#115

Post by embedding-shape »

JeremyNT wrote: post #103
rzmmm wrote: post #3

Quote: "My personal conclusion can however not end up with anything else than that the big hype around this model so far was primarily marketing. I see no evidence that this setup finds issues to any particular higher or more advanced degree than the other tools have done before Mythos. Maybe this model is a little bit better, but even if it is, it is not better to a degree that seems to make a significant dent in co…

This is roughly what I was assuming but of course the big caveat here is that they were already using the existing LLM driven tooling on an extensively audited codebase. So while anthropic's marketing may be hype there just wasn't much left to find, a point he makes in the blog post. Whether it's a big step forward for other kinds of projects is difficult to tell, but this highlights that everybody should be using AI…

None of those other LLM tooling made the claims they're too dangerous to be released and used though, unlike Anthropic did with Mythos.

What it highlights, is that Mythos doesn't seem so much better than other LLM driven tooling at finding security issues, which was the strongest claim Anthropic made in the first place.

Re: Mythos Finds a Curl Vulnerability

#116

Post by keybored »

bigcat12345678 wrote: post #56
rzmmm wrote: post #3

Quote: "My personal conclusion can however not end up with anything else than that the big hype around this model so far was primarily marketing. I see no evidence that this setup finds issues to any particular higher or more advanced degree than the other tools have done before Mythos. Maybe this model is a little bit better, but even if it is, it is not better to a degree that seems to make a significant dent in co…

My guess: Marketing is not intentional. Evidences: 10 years ago, when I interviewed Baidu AI with Andrew Ng and Dario, Dario is the kind of person is pure-hearted to the point being ideological. Given Dario's successful career so far, that essence has gradually grown into a conviction, and surrounded by a purposely built team which amplifies his ideology. Humans are very convenient creature, a rare few small fraction…

This is marketing.

Re: Mythos Finds a Curl Vulnerability

#117

Post by zeroCalories »

red75prime wrote: post #99
teiferer wrote: post #69

Earlier quoted context omitted.

Given how much money is on the line, it would be gross negligence if anything came publicly out of the CEO's mouth or is otherwise published by the company that's not marketing.

The question is whether they need to massage the results for them to be marketable.

Sometimes you gotta let people know how awesome you are. The real question is if you're misrepresenting yourself(all marketing, no substance).

Re: Mythos Finds a Curl Vulnerability

#118

Post by windexh8er »

bigcat12345678 wrote: post #56
rzmmm wrote: post #3

Quote: "My personal conclusion can however not end up with anything else than that the big hype around this model so far was primarily marketing. I see no evidence that this setup finds issues to any particular higher or more advanced degree than the other tools have done before Mythos. Maybe this model is a little bit better, but even if it is, it is not better to a degree that seems to make a significant dent in co…

My guess: Marketing is not intentional. Evidences: 10 years ago, when I interviewed Baidu AI with Andrew Ng and Dario, Dario is the kind of person is pure-hearted to the point being ideological. Given Dario's successful career so far, that essence has gradually grown into a conviction, and surrounded by a purposely built team which amplifies his ideology. Humans are very convenient creature, a rare few small fraction…

Marketing is always intentional at this scale. If you think Anthropic didn't put a lot of time and effort into Glasswing as a marketing effort I think you're misunderstanding how these organizations work and how they win.

Re: Mythos Finds a Curl Vulnerability

#119

Post by hug »

embedding-shape wrote: post #115
JeremyNT wrote: post #103

Earlier quoted context omitted.

This is roughly what I was assuming but of course the big caveat here is that they were already using the existing LLM driven tooling on an extensively audited codebase. So while anthropic's marketing may be hype there just wasn't much left to find, a point he makes in the blog post. Whether it's a big step forward for other kinds of projects is difficult to tell, but this highlights that everybody should be using AI…

None of those other LLM tooling made the claims they're too dangerous to be released and used though, unlike Anthropic did with Mythos. What it highlights, is that Mythos doesn't seem so much better than other LLM driven tooling at finding security issues, which was the strongest claim Anthropic made in the first place.

Actually, OpenAI made a similar claim about one of their GPT models a while ago…

Funnily enough that was while Dario Amodei was their research director.

Re: Mythos Finds a Curl Vulnerability

#120

Post by jabwd »

stingraycharles wrote: post #76
johnbarron wrote: post #59

Earlier quoted context omitted.

>> Anthropic using marketing to convince people their models are more advanced, better built, or that AI is a threat that needs to be regulated because only they have the answer? I’m shocked. I remember when OpenAI was saying GPT-2 was too dangerous to release.

I remember when there was a guy at Google years a few years ago that was convinced that they had an internal, sentient creature in their labs (I think maybe 4 years ago?) If I’m not mistaken, after the media cycle, he lost his job for breaking confidentiality. That was the opposite of marketing, Google really didn’t get how to turn this into a product until ChatGPT happened.

They most likely understood that it wasn't viable for anything. OpenAI just yolo'd it and now we're dealing with the fallout. I'm fairly certain that any management layer at google isn't going to say yes to "invest 5 billion to make 10 million" scheme that OpenAI, Anthropic, are currently running.
Post reply on HN