191–200 of 372 posts
Earlier quoted context omitted.
No, they didn't. They distinguished it, when presented with it. Wildly different problem.
Yeah. And it is totally depressing that this article got voted to the top of the front page. It means people aren’t capable of this most basic reasoning so they jumped on the “aha! so the mythos announcement was just marketing!!”
Earlier quoted context omitted.
No, they didn't. They distinguished it, when presented with it. Wildly different problem.
Yeah. And it is totally depressing that this article got voted to the top of the front page. It means people aren’t capable of this most basic reasoning so they jumped on the “aha! so the mythos announcement was just marketing!!”
Earlier quoted context omitted.
> because small models found the same vulnerability. With a ton of extra support. Note this key passage: >We isolated the vulnerable svc_rpc_gss_validate function, provided architectural context (that it handles network-parsed RPC credentials, that oa_length comes from the packet), and asked eight models to assess it for security vulnerabilities. Yeah it can find a needle in a haystack without false positives, if you…
The benefit here is reducing the time to find vulnerabilities; faster than humans, right? So if you can rig a harness for each function in the system, by first finding where it’s used, its expected input, etc, and doing that for all functions, does it discover vulnerabilities faster than humans? Doesn’t matter that they isolated one thing. It matters that the context they provided was discoverable by the model.
Earlier quoted context omitted.
Also, what is $20,000 today can be $2000 next year. Or $20... See e.g. https://epoch.ai/data-insights/llm-inference-price-trends/
Or $200,000 for consumers when they have to make a profit
So there are two competing narratives: 1. Mythos uniquely is able to find vulnerabilities that other LLMs cannot practically. 2. All LLMs could already do this but no one tried the way anthropic did. The truth is one of these. And it comes down whether the comparison is apples to apples. Since we don't know the exact specifics of how either tests were performed, we lack a way of knowing absolutely. So I guess, like s…
https://sean.heelan.io/2025/05/22/how-i-used-o3-to-find-cve-...
Earlier quoted context omitted.
The citation is the Anthropic writeup.
They did not say what you are saying… > If you try to automate a small model to look for vulnerabilities over 10,000 files, it's going to say there are 9,500 vulns.
The "9500" quote is my conjecture of what might happen if they fix their approach, but the burden of proof is definitely not on me to actually fix their writeup and spend a bunch of money to run a new eval! They are the ones making a claim on shaky ground, not me.