Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Six curl CVEs after OpenAI and Anthropic came back with zero

aisle.com

Post reply on HN
Print view View original

41–50 of 68 posts

Re: Six curl CVEs after OpenAI and Anthropic came back with zero

#41

Post by bluGill »

fc417fc802 wrote: post #39
bluGill wrote: post #32

Earlier quoted context omitted.

Curl has been scanned by mythos and several other AI tools several times over the past year already. Mythos found nothing this round, but when it was first released it found issues which have been fixed - and several other AIs had already scanned curl for issues which had been fixed by then. We can say that this is a useful tool, but is it better or worse than the others - there is no way to make that conclusion. Edi…

I don't understand your objection. If tool A says "job is done" and tool B says "found N additional tasks that need doing" how does the history matter? B is turning up things that A isn't thus B is performing better. They both had access to the same inputs here. I suppose it's interesting to wonder if B would have turned up issue X which A previously found. But that seems largely academic to me. There is a code base…

The "Better" is where I have an issue. There isn't enough information here to say one is better than the other. Both are useful tools is clear.

Re: Six curl CVEs after OpenAI and Anthropic came back with zero

#45

Post by rcxdude »

matherial wrote: post #25
bch wrote: post #15

Earlier quoted context omitted.

Also sounds incredibly compute intensive.

Setting a swarm of agents loose for hours to look for software vulnerabilities is far more compute-expensive than fuzzing. The industry has never thrown this kind of compute resources at pure fuzzing, in part because you can't get much VC money for that.

You also quickly get into diminishing returns with fuzzing. Generally a bug is either going to be found relatively quickly with a given fuzzing approach or it's going to be nearly impossible to find. You're usually better coming up with more intelligent fuzzing approaches than you are just dumping compute into it.

Re: Six curl CVEs after OpenAI and Anthropic came back with zero

#46

Post by goobreee »

_pdp_ wrote: post #35

I like the looks of Aisle and what they stand for... That being said you cannot compare a model with a specialised harness. These are two completely different things. Am I missing something?

i think a key missing part is that an LLM on its own can't find vulnerabilities, so it's always an AI + harness. even mythos, when used for finding zero-days, is using an actually surprisingly heavy handed and expensive scaffold. they literally make it run in parallel on ±all files and ask "what's wrong with this?". here from the mythos technical blog post [1]:

> To increase efficiency, instead of processing literally every file for each software project that we evaluate, we first ask Claude to rank how likely each file in the project is to have interesting bugs on a scale of 1 to 5. A file ranked “1” has nothing at all that could contain a vulnerability (for instance, it might just define some constants). Conversely, a file ranked “5” might take raw data from the Internet and parse it, or it might handle user authentication. We start Claude on the files most likely to have bugs and go down the list in order of priority.

So they process it in parallel, but AI-rank them based on vuln-likelihood first = exhaustive search with a heuristic filtering first

[1] https://www.anthropic.com/research/mythos-preview

Re: Six curl CVEs after OpenAI and Anthropic came back with zero

#47

Post by syou1024 »

bch wrote: post #15
tux3 wrote: post #8

Earlier quoted context omitted.

The homepage says something about AI guided fuzzing based on libfuzzer or AFL. Looks like they have the LLMs identify a bunch of interesting functions to test, generate some test harnesses, and then sort through the fuzzer findings at a high level, which sounds like a pretty good idea.

Also sounds incredibly compute intensive.

[dead]

Re: Six curl CVEs after OpenAI and Anthropic came back with zero

#49

Post by grumpy-swe-9000 »

bryanlarsen wrote: post #29
vorticalbox wrote: post #27

Earlier quoted context omitted.

It defaults to gpt5.4 nano https://github.com/weareaisle/nano-analyzer/blob/main/scan.p...

A repo named "nano-analyzer" unsurprisingly uses gpt5.4 nano. I doubt their "pay them money" version uses nano.

I am pretty sure that the nano-analyzer is just a limited open source demo of their "System over Model" thesis from https://aisle.com/blog/system-over-model-zero-day-discovery-..., not the main product.

Re: Six curl CVEs after OpenAI and Anthropic came back with zero

#50

Post by grumpy-swe-9000 »

tosti wrote: post #37

One does not "discover" a CVE like this. To discover a CVE would mean you searched for a particular piece of software and found it vulnerable according to the NVD. That's not a novel discovery by any means. What they did is they found bugs and that they were exploitable in certain edge cases. As the bugs turned out to be vulnerabilities, they were assigned a CVE in the NVD with low severity. IMHO Aisle stockedpiled t…

What would the correct terminology actually be here I wonder?
Post reply on HN