Six curl CVEs after OpenAI and Anthropic came back with zero
1–10 of 68 posts
Re: Six curl CVEs after OpenAI and Anthropic came back with zero
#2That's bragging rights correctly earned, i think! As marketing-y as this post is, definitely something to keep an eye on.
Re: Six curl CVEs after OpenAI and Anthropic came back with zero
#3Wow, this announcement is good content marketing.
Don't get me wrong, it's interesting. But there is no technical discussion as to how they did it. It's simply: we did it and Mythos and Codex didn't.
It's good to know that it's possible, but I'd have already expected it. Put a base model versus a base model + harness + whatever else, and yea, if you do it right then you have a better system to find vulnerabilities.
> We then ran AISLE's autonomous AI system against curl.
They don't even mention what models the use under the hood. It wouldn't surprise me if they are from Anthropic and OpenAI.
Re: Six curl CVEs after OpenAI and Anthropic came back with zero
#4Wow, this announcement is good content marketing. Don't get me wrong, it's interesting. But there is no technical discussion as to how they did it. It's simply: we did it and Mythos and Codex didn't. It's good to know that it's possible, but I'd have already expected it. Put a base model versus a base model + harness + whatever else, and yea, if you do it right then you have a better system to find vulnerabilities. >…
> what models the use under the hood
Presumably their own, wouldn’t they?
Re: Six curl CVEs after OpenAI and Anthropic came back with zero
#5We had a few AISLE-generated security reports, and the signal to noise was reasonably good.
The most notable bug/exploit their scanner found was: https://gitlab.com/nbdkit/libnbd/-/commit/e50bbd2681117c2dd8...
The tool basically had to chain two exploits together to reach this. It also came up with a patch to fix which was fairly sensible (but I ended up editing it further for clarity).
Re: Six curl CVEs after OpenAI and Anthropic came back with zero
#6Good marketing and definitive proof that local (read: on-prem & air-gapped) models with correct context and tools are good enough to perform on par and above SOTA cloud hosted solutions.
We have seen this point many times before with different technologies. The first computers at university were big and expensive, same as this machine. Give it a few years and this functionality will be a commodity.
Re: Six curl CVEs after OpenAI and Anthropic came back with zero
#7Wow, this announcement is good content marketing. Don't get me wrong, it's interesting. But there is no technical discussion as to how they did it. It's simply: we did it and Mythos and Codex didn't. It's good to know that it's possible, but I'd have already expected it. Put a base model versus a base model + harness + whatever else, and yea, if you do it right then you have a better system to find vulnerabilities. >…
> what models the use under the hood Presumably their own, wouldn’t they?
You mean their own trained models, or do you think it's an open source model that they fine-tuned? If they use their own, I'd guess it's the latter.
Re: Six curl CVEs after OpenAI and Anthropic came back with zero
#8Wow, this announcement is good content marketing. Don't get me wrong, it's interesting. But there is no technical discussion as to how they did it. It's simply: we did it and Mythos and Codex didn't. It's good to know that it's possible, but I'd have already expected it. Put a base model versus a base model + harness + whatever else, and yea, if you do it right then you have a better system to find vulnerabilities. >…
The homepage says something about AI guided fuzzing based on libfuzzer or AFL. Looks like they have the LLMs identify a bunch of interesting functions to test, generate some test harnesses, and then sort through the fuzzer findings at a high level, which sounds like a pretty good idea.
Re: Six curl CVEs after OpenAI and Anthropic came back with zero
#9Wow, this announcement is good content marketing. Don't get me wrong, it's interesting. But there is no technical discussion as to how they did it. It's simply: we did it and Mythos and Codex didn't. It's good to know that it's possible, but I'd have already expected it. Put a base model versus a base model + harness + whatever else, and yea, if you do it right then you have a better system to find vulnerabilities. >…
The homepage says something about AI guided fuzzing based on libfuzzer or AFL. Looks like they have the LLMs identify a bunch of interesting functions to test, generate some test harnesses, and then sort through the fuzzer findings at a high level, which sounds like a pretty good idea.
Thanks for figuring that out. Sort of sounds like AI programming programs to find vulnerabilities, of which fuzzing is one of the proven techniques to do it.
Re: Six curl CVEs after OpenAI and Anthropic came back with zero
#10Wow, this announcement is good content marketing. Don't get me wrong, it's interesting. But there is no technical discussion as to how they did it. It's simply: we did it and Mythos and Codex didn't. It's good to know that it's possible, but I'd have already expected it. Put a base model versus a base model + harness + whatever else, and yea, if you do it right then you have a better system to find vulnerabilities. >…
Their system can run with various models, they go into more details in this article.
https://aisle.com/blog/system-over-model-zero-day-discovery-...