Standard disclosure rules should apply, give security stake holders 90-days of advance access, then release the model.
By that logic though the model would release 90 days from the last vulnerability it finds, so never.
Has Mythos just broken the deal that kept the internet safe?
41–50 of 65 posts
Re: Has Mythos just broken the deal that kept the internet safe?
#42> According to Anthropic, Mythos Preview successfully generates a working exploit for Firefox's JS shell in 72.4% of trials Why are AI people so dramatic? Ok, there is yet another JS sandbox escape - not the first one, not the last one. It will be patched, and the bar will be raised for a bit... at least until the next exploit is found. If anything, AI will make _weaponized_ exploits less likely. Before, one had to f…
Ding ding ding, and this is why you are hearing about it. It is marketing for enterprise to pay a premium for the next model, with maybe a wakeup call to enforcement agencies as well (which is also marketing). Codegen for many companies is much less continuous. Security is always on, and always a motivator.
This whole thing has just been a huge PR stunt the whole time. Even the original leak of the blog post was just more fuel to the hype.
Re: Has Mythos just broken the deal that kept the internet safe?
#43> According to Anthropic, Mythos Preview successfully generates a working exploit for Firefox's JS shell in 72.4% of trials Why are AI people so dramatic? Ok, there is yet another JS sandbox escape - not the first one, not the last one. It will be patched, and the bar will be raised for a bit... at least until the next exploit is found. If anything, AI will make _weaponized_ exploits less likely. Before, one had to f…
You’re asking why people are being “dramatic” about an automated system that can do what highly specialized experts get paid hundreds of thousands of dollars to do? It’s just fascinating to see how AI’s accomplishments are being systematically downplayed. I guess when an AI proves that P!=NP, I’m going to read on this forum “so what, mathematicians prove conjectures all the time, and also, we pretty much always knew…
I am sceptical because AI companies, and anthropic in particular, like to overplay their achievements and build undeserved hype. I also don't understand all the caveats (maybe official announcement is more clear what this really means).
But yeah, if their model can reliably write an exploit for novel bugs (starting from a crash, not a vulnerable line of code) then it's very significant. I guess we'll see, right?
edit: Actually the original post IS dramatic: "Has Mythos just broken the deal that kept the internet safe? For nearly 20 years the deal has been simple: you click a link, arbitrary code runs on your device, and a stack of sandboxes keeps that code from doing anything nasty". Browser exploits have existed before, and this capability helps defenders as much as it helps attackers, it's not like JS is going anywhere.
Re: Has Mythos just broken the deal that kept the internet safe?
#44Earlier quoted context omitted.
Further, Opus identified most of the vulnerabilities itself already. It just couldn’t exploit them. Mythos seems much, much more creative and self directed, but I’m not yet convinced the core capabilities are significantly higher than what’s possible today. The full price of finding the vulnerabilities was also something like $20k. That’s a price point that brings a skilled professional in to accomplish the same task…
Remember, that's the most expensive this capability will ever be.
If it's model is opened up and can run on commodity hardware. Otherwise price could go up as RAM and silicon prices climb.
Re: Has Mythos just broken the deal that kept the internet safe?
#45I see this as a Brandolini's Law 2.0, a software supplemental really. Where-as before it was:
> The amount of energy needed to refute bullshit is an order of magnitude bigger than that needed to produce it.
Now the energy needed to secure against exploits is orders of magnitude bigger than the effort needed to secure it.
The combination of deep expertise + infinite patience of the LLM meeting the vastly increasing surface of software has a certain apocalyptic chaos gods ruin to it all, just as well known bias for mistruth to unfairly propogate itself bedevils this good planet.
Re: Has Mythos just broken the deal that kept the internet safe?
#46I tried to read the article and what I got out of it was that the author believes that the deal that keeps the internet safe is that we just don't try to break it hard enough. Ignoring all the state actors who do that all the time. Seems something of a unusual take on the state of the world
Really? I think that's pretty much accurate. If you've ever visited a website whose authors you don't know and trust, you've exposed yourself to potential attacks and trusted in sandboxing to keep your computer safe.
Re: Has Mythos just broken the deal that kept the internet safe?
#47>Anthropic just launched a model so good it scapes every know sandboxed. No, they launched a card with that capability written on.
And the companies such as Google and Nvidia are just happy to trust them and lent their names to Anthropic because? Maybe a big conspiracy?
Re: Has Mythos just broken the deal that kept the internet safe?
#48> According to Anthropic, Mythos Preview successfully generates a working exploit for Firefox's JS shell in 72.4% of trials Why are AI people so dramatic? Ok, there is yet another JS sandbox escape - not the first one, not the last one. It will be patched, and the bar will be raised for a bit... at least until the next exploit is found. If anything, AI will make _weaponized_ exploits less likely. Before, one had to f…
> Before, one had to find a talented person, and get pretty lucky too. If this AI is as good as promised, you can have dependabot-style exploit finder running 24/7 for the 1/10th cost of a single FTE Not you. EVERYONE doing ANY kind of software will have to, because else attacker can just pick and choose targets to point their exploit-bot
Not "ANY" kind of software, only the software that handles untrusted data in a non-trivial way. A lot of software, like local tools, does not.
Re: Has Mythos just broken the deal that kept the internet safe?
#49>Anthropic just launched a model so good it scapes every know sandboxed. No, they launched a card with that capability written on.
And the companies such as Google and Nvidia are just happy to trust them and lent their names to Anthropic because? Maybe a big conspiracy?
token usage go up, said companies $TICKER price go up, shareholder value delivered.
when shareholders are basically the same, and this companies have a legal obligation to fulfill their interests...is it a conspiracy? shareholders certainly conspire to achieve their goals, smarty
Re: Has Mythos just broken the deal that kept the internet safe?
#50Earlier quoted context omitted.
All software has bugs. What this tells me is that the actors with the best models (and Anthropic apparently has one so good and expensive it is outstripping compute supply) they will find the exploits first and probably the ones that are hardest to find So yeah, dependabot, but the richest actors will have the best bits and they probably won’t share the ones they can find that nobody else’s models can
> What this tells me is that the actors with the best models (and Anthropic apparently has one so good and expensive it is outstripping compute supply) they will find the exploits first and probably the ones that are hardest to find Presumably we would not give the AI models to the "good guys" because then they would also find and patch these vulnerabilities?
Someone's "good guys" are just someone "bad guys". Access to a valuable resource/tool that provides some sort of power and utility will be just another contended item.