Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Mythos Finds a Curl Vulnerability

daniel.haxx.se

Post reply on HN
Print view View original

51–60 of 298 posts

Re: Mythos Finds a Curl Vulnerability

#51

Post by therealpygon »

rzmmm wrote: post #3

Quote: "My personal conclusion can however not end up with anything else than that the big hype around this model so far was primarily marketing. I see no evidence that this setup finds issues to any particular higher or more advanced degree than the other tools have done before Mythos. Maybe this model is a little bit better, but even if it is, it is not better to a degree that seems to make a significant dent in co…

Anthropic using marketing to convince people their models are more advanced, better built, or that AI is a threat that needs to be regulated because only they have the answer? I’m shocked.

More seriously, so far I haven’t seen much indication that Mythos is more than Opus with a security focused code analysis harness. That said, the fact it can find these bugs in an automated fashion is the more important takeaway outside of the hype.

I’m curious what the error rate is on the detections, because none of that means much if it is wrong 90% of the time and we are only hearing about the examples that are useful marketing.

Re: Mythos Finds a Curl Vulnerability

#53

Post by croon »

bilekas wrote: post #34
cakealert wrote: post #22

Earlier quoted context omitted.

> No, based on cURL's history, it really seems like they would love to have found a really novel bug. You just confirmed that you didn't read the article. "Eventually, I was instead offered that someone else, who has access to the model, could run a scan and analysis on curl for me using Mythos and send me a report."

I'm not sure how that proves I didn't read the article ?

Someone external to the curl team ran the test. If that third party found a severe CVE that they could use across all the global curl attack surface, and did not disclose it back to the curl team, the third party could keep using the exploit until discovered independently.

Re: Mythos Finds a Curl Vulnerability

#54

Post by dotancohen »

Hamuko wrote: post #17
dotancohen wrote: post #15

Earlier quoted context omitted.

Perhaps he was dictating. Does it say anything else? Just 'Aaaarggghhhh'?

Doubt it considering that Daniel Stenberg is Swedish. English dictation when you speak English as a second language with an accent is quite annoying.

I understand completely. You don't want to know what the machine produced, when I asked it for "a new display".

Re: Mythos Finds a Curl Vulnerability

#55

Post by casey2 »

ahofmann wrote: post #2

Putting on my tinfoil-hat: Sooo, the guy who runs the test and delivers the report could just have removed the more interesting bugs and delivered those to any three letter agency?

curl's source is public so what would be the gain in the rigmarole? Now if the prompt was "create a patch that inserts a zero-day while fixing a bug" that would be impressive.

Re: Mythos Finds a Curl Vulnerability

#56

Post by bigcat12345678 »

rzmmm wrote: post #3

Quote: "My personal conclusion can however not end up with anything else than that the big hype around this model so far was primarily marketing. I see no evidence that this setup finds issues to any particular higher or more advanced degree than the other tools have done before Mythos. Maybe this model is a little bit better, but even if it is, it is not better to a degree that seems to make a significant dent in co…

My guess:

Marketing is not intentional.

Evidences: 10 years ago, when I interviewed Baidu AI with Andrew Ng and Dario, Dario is the kind of person is pure-hearted to the point being ideological. Given Dario's successful career so far, that essence has gradually grown into a conviction, and surrounded by a purposely built team which amplifies his ideology.

Humans are very convenient creature, a rare few small fraction of them are no doubt the master of convenience: they morph their mental manifold without a hint of contradiction in their own mental mechanisms.

Re: Mythos Finds a Curl Vulnerability

#57

Post by redditor98654 »

mohsen1 wrote: post #33
dyauspitr wrote: post #27

Earlier quoted context omitted.

Having never used Claude and only Codex, does Claude actually say “this is too involved” as a response to a prompt?

Yes it does. Usually after hours of working and not getting results

I am curious, what kind of work do you use Claude for that sometimes requires hours of working. In my case, I have never seen it go off for more than 10 mins and even that is very rare.

Re: Mythos Finds a Curl Vulnerability

#59

Post by johnbarron »

therealpygon wrote: post #51
rzmmm wrote: post #3

Quote: "My personal conclusion can however not end up with anything else than that the big hype around this model so far was primarily marketing. I see no evidence that this setup finds issues to any particular higher or more advanced degree than the other tools have done before Mythos. Maybe this model is a little bit better, but even if it is, it is not better to a degree that seems to make a significant dent in co…

Anthropic using marketing to convince people their models are more advanced, better built, or that AI is a threat that needs to be regulated because only they have the answer? I’m shocked. More seriously, so far I haven’t seen much indication that Mythos is more than Opus with a security focused code analysis harness. That said, the fact it can find these bugs in an automated fashion is the more important takeaway ou…

>> Anthropic using marketing to convince people their models are more advanced, better built, or that AI is a threat that needs to be regulated because only they have the answer? I’m shocked.

I remember when OpenAI was saying GPT-2 was too dangerous to release.

Re: Mythos Finds a Curl Vulnerability

#60

Post by AntiUSAbah »

flohofwoe wrote: post #40
AntiUSAbah wrote: post #12

There is always marketing involved and people should be able to put marketing into perspective. Also curl in this regard is a open source project, relativly small but critical, well known and used everywhere. Besides image libraries, tools like curl or sudo, su, passwd, etc. would also be my first try. Mythos is still not known at all what it can do. What does it mean from cost and benchmark pov to have a 10 Trillion…

> Nonetheless, the fact that LLMs got significant better in finding this, better than humans, started to happen half a year ago? *rolls eyes* regular static analyzers also have been "better than humans" for decades, being better than a human at a specific mechanical task really doesn't mean much. The interesting new thing is the type of potential "fuzzy bugs" described in the article that LLMs are able to identify (a…

You don't have to dismantle a comment on a microlevel.

It has been clear for ages that certain type of bugs or issues are better solved from software.

But there was still plenty of things a proper SecOps Person would be able to find with help from tooling which automatic tooling wouldn't find.

Taking a limited amount of resources and focusing on the critical things.

I do think this is gone now. Same with Threat modeling etc.

Post reply on HN