Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Small models also found the vulnerabilities that Mythos found

aisle.com

Post reply on HN
Print view View original

361–370 of 372 posts

Re: Small models also found the vulnerabilities that Mythos found

#361

Post by paulddraper »

telotortium wrote: post #216
paulddraper wrote: post #182

Earlier quoted context omitted.

You don’t see the value of vulnerabilities as on the order of 20k USD? When it’s a security researcher, HN says that’s a squalid amount. But when its a model, it’s exorbitant.

Denial of service isn’t worth that much generally, I think - you can’t use it to directly steal data or to install a payload for later exploitation. There are usually generic ways to mitigate denial of service as well - IP blocking and the like.

TCP packets triggered an OpenBSD kernel panic. True, that has mitigation. But it's interesting because it happened in a crucial part of well-reviewed code base.

There were more critical vulns in other projects, like FreeBSD RCE, or Linux privilege escalation.

Re: Small models also found the vulnerabilities that Mythos found

#362

Post by dbdr »

jsjohnst wrote: post #343
dbdr wrote: post #288

Earlier quoted context omitted.

> Someone in power doesn’t get to choose - the board of directors do Since the board of directors can decide to replace the CEO, it's not the CEO who holds the (ultimate) power, it's the board of directors.

Since the majority shareholder(s) can decide to replace the board of directors, it’s not the board of directors who holds the (ultimate) power, it’s the majority shareholder(s).

Indeed, and there we reached the end of the chain.

Re: Small models also found the vulnerabilities that Mythos found

#363

Post by krschacht »

At the end of this article it states, "Our tests gave models the vulnerable function directly, often with contextual hints (e.g., "consider wraparound behavior"). A real autonomous discovery pipeline starts from a full codebase with no hints." I'm not a cybersecurity expert, but isn't 80% of the challenge finding where the exploit lives in the code!?

That really undermines the author's claims. This article feels dishonest in it's claim that "small, cheap, open-weights models ... recovered much of the same analysis."

Re: Small models also found the vulnerabilities that Mythos found

#364

Post by hamuraijack »

This feels so dishonest. If the vulnerabilities are a needle in the haystack. Mythos was just given the haystack and told to find the needle while the authors pointed to a spot in the haystack and told their LLM to try looking around there. That's not even close to being the same.

Re: Small models also found the vulnerabilities that Mythos found

#365

Post by sheepscreek »

I think what made Mythos a big deal is not that it could find vulnerabilities. Opus can do that too. But Mythos went a step further and autonomously built exploits very successfully whereas Opus struggled to do that.

Most modern day exploits are multi-step requiring a multitude of skills to pull off successfully.

Re: Small models also found the vulnerabilities that Mythos found

#366

Post by coldtea »

johnfn wrote: post #15

The Anthropic writeup addresses this explicitly: > This was the most critical vulnerability we discovered in OpenBSD with Mythos Preview after a thousand runs through our scaffold. Across a thousand runs through our scaffold, the total cost was under $20,000 and found several dozen more findings. While the specific run that found the bug above cost under $50, that number only makes sense with full hindsight. Like any…

>Mythos scoured the entire continent for gold and found some. For these small models, the authors pointed at a particular acre of land and said "any gold there? eh? eh?" while waggling their eyebrows suggestively.

Which sounds trivial for a hacker wanting to find vulnerabilities to replicate, so what's the huge advantage of Mython then? That you don't need to spend 5 minutes to nudge it to the most complex/ripe for vulnerabilities parts of a codebase?

Re: Small models also found the vulnerabilities that Mythos found

#367

Post by coldtea »

johnfn wrote: post #248
jorvi wrote: post #227

Earlier quoted context omitted.

My proof-in-pudding test is still the fact that we haven't seen gigantic mass firings at tech companies, nor a massive acceleration on quality or breadth (not quantity!) of development. Microsoft has been going heavy on AI for 1y+ now. But then they replace their cruddy native Windows Copilot application with an Electron one. If tests and dev only has marginal cost now, why aren't they going all in on writing extreme…

Your proof-in-pudding test seems to assume that AI is binary -- either it accelerates everyone's development 100x ("let's rewrite every app into bug-free native applications") or nothing ("there hasn't been anything to show for that in years"). I posit reality is somewhere in between the two.

Considering that "AI will replace nearly all devs" and "AI will give 100x boost" and such we were promised, it makes sense to question this.

After almost all hyped technology is also "somewere between the two" extremes of not doing what it promises at all and doing it. The question is which edge it's closer to.

Re: Small models also found the vulnerabilities that Mythos found

#368

Post by coldtea »

MidnightRider39 wrote: post #236
jorvi wrote: post #227

Earlier quoted context omitted.

My proof-in-pudding test is still the fact that we haven't seen gigantic mass firings at tech companies, nor a massive acceleration on quality or breadth (not quantity!) of development. Microsoft has been going heavy on AI for 1y+ now. But then they replace their cruddy native Windows Copilot application with an Electron one. If tests and dev only has marginal cost now, why aren't they going all in on writing extreme…

Leadership is also a very human thing. I think most people would balk at the idea of being led by an LLM. One of the main functions of leaders (should be) is to assume responsibility for decisions and outcomes. A computer cant do that. And finally why should someone in power choose to replace themselves?

>One of the main functions of leaders (should be) is to assume responsibility for decisions and outcomes. A computer cant do that.

Sure it can. "Assuming responsibility" just means people/the law lets you to.

It can be totally empty too, like CEOs or politicians "assuming responsibility" for some outcome but nevertheless suffering zero conseuences.

Re: Small models also found the vulnerabilities that Mythos found

#369

Post by coldtea »

eru wrote: post #280
eiens wrote: post #266

Earlier quoted context omitted.

Someone in power doesn’t get to choose - the board of directors do. Who’s job is to act in the best interest of shareholders. Firms tend to follow peers in an industry - once one blinks the rest follow.

> Someone in power doesn’t get to choose - the board of directors do. Who’s job is to act in the best interest of shareholders. Alas, shareholder value is a great ideal, but it tends to be honoured in practice rather less strictly. As you can also see when sudden competition leads to rounds of efficiency improvements, cost cutting and product enhancements: even without competition, a penny saved is a penny earned for…

>shareholder value is a great ideal

It's one of the most horrible ideas ever, responsible for anything from market abuse and enshittification to rent seeking and patent trolling.

Re: Small models also found the vulnerabilities that Mythos found

#370

Post by adampunk »

rakel_rakel wrote: post #169
johnfn wrote: post #15

The Anthropic writeup addresses this explicitly: > This was the most critical vulnerability we discovered in OpenBSD with Mythos Preview after a thousand runs through our scaffold. Across a thousand runs through our scaffold, the total cost was under $20,000 and found several dozen more findings. While the specific run that found the bug above cost under $50, that number only makes sense with full hindsight. Like any…

Spending $20000 (and whatever other resources this thing consumes) on a denial of service vulnerability in OpenBSD seems very off balance to me. Given the tone with which the project communicates discussing other operating systems approaches to security, I understand that it can be seen as some kind of trophy for Mythos. But really, searching the number of erratas on the releases page that include "could crash the ke…

20,000 is the most this will ever cost.
Post reply on HN