Those 3 letter agencies are going to see their stash of 0-days dwindle so hard.
Their stash will explode. LLMs can do this on binaries just the same, and there's a lot more closed than open source SW out there.
Claude Code Found a Linux Vulnerability Hidden for 23 Years
251–260 of 303 posts
Re: Claude Code Found a Linux Vulnerability Hidden for 23 Years
#252Every time I read these titles, I wonder if people are for some reason pushing the narrative that Claude is way smarter than it really is, or if I'm using it wrong. They want me to code AI-first, and the amount of hallucinations and weird bugs and inconsistencies that Claude produces is massive. Lots of code that it pushes would NOT have passed a human/human code review 6 months ago.
It's always the inconsistencies which amaze me, from the article: > I have so many bugs in the Linux kernel that I can’t report because I haven’t validated them yet You have "so many?" Are they uncountable for some reason? You "haven't validated" them? How long does that take? > found a total of five Linux vulnerabilities And how much did it cost you in compute time to find those 5? These articles are always fantasti…
You are suspicious because you probably haven't worked anywhere that's AI-first. Anyone that's worked at a modern tech company will find this absolutely believable.
Like what, you expect Nicholas to test each vuln when he has more important work to do (ie his actual job?)
Re: Claude Code Found a Linux Vulnerability Hidden for 23 Years
#253I'm interested in the implications for the open source movement, specifically about security concerns. Anyone know is there has been a study about how well Claude Code works on closed source (but decompiled) source?
Definitely not my wheelhouse, but I would expect it to be considerably worse. Simply because the source code contains names that were intended to communicate meaning in a way that the LLM is specifically trained to understand (i.e., by choosing identifier names from human natural language, choosing those names to scan well when interspersed into the programming language grammar, including comments etc.). At least if…
I've cut-and-pasted some assembly code into the free version of ChatGPT to reverse engineer some old binaries and its ability to find meaning was just scary.
Re: Claude Code Found a Linux Vulnerability Hidden for 23 Years
#254I'm interested in the implications for the open source movement, specifically about security concerns. Anyone know is there has been a study about how well Claude Code works on closed source (but decompiled) source?
It would be much more interesting/efficient if the LLM had tokens for machine instructions so extracting instructions would be done at tokenizing phase, not by calling objdump. But I guess I'm not the first one to have that idea. Any references to research papers would be welcome.
As an experiment, I just now took a random section of a few hundreds bytes (as a hexdump) from the /bin/ls executable and pasted them into ChatGPT.
I don't know if it's correct, but it speculated that it's part of a command line processor: https://chatgpt.com/share/69d19e4f-ff2c-83e8-bc55-3f7f5207c3...
Now imagine how much more it could have derived if I had given it the full executable, with all the strings, pointers to those strings and whatnot.
I've done some minor reverse engineering of old test equipment binaries in the past and LLMs are incredible at figuring out what the code is doing, way better than the regular way of Ghidra to decompile code.
Re: Claude Code Found a Linux Vulnerability Hidden for 23 Years
#255Pasting a big batch of new code and asking Claude "what have I forgotten? Where are the bugs?" is a very persuasive on-ramp for developers new to AI. It spots threading & distributed system bugs that would have taken hours to uncover before, and where there isn't any other easy tooling. I bet there's loads of cryptocurrency implementations being pored over right now - actual money on the table.
I like biasing it towards the fact that there is a bug, so it can't just say "no bugs! all good!" without looking into it very hard. Usually I ask something like this: "This code has a bug. Can you find it?" Sometimes I also tell it that "the bug is non-obvious" Which I've anecdotally found to have a higher rate of success than just asking for a spot check
Do you not run into too many false positives around "ah, this thing you used here is known to be tricky, the issue is..."
I've seen that when prompting it to look for concurrency issues vs saying something more like "please inspect this rigorously to look for potential issues..."
Re: Claude Code Found a Linux Vulnerability Hidden for 23 Years
#256Earlier quoted context omitted.
I like biasing it towards the fact that there is a bug, so it can't just say "no bugs! all good!" without looking into it very hard. Usually I ask something like this: "This code has a bug. Can you find it?" Sometimes I also tell it that "the bug is non-obvious" Which I've anecdotally found to have a higher rate of success than just asking for a spot check
Do you not run into too many false positives around "ah, this thing you used here is known to be tricky, the issue is..." I've seen that when prompting it to look for concurrency issues vs saying something more like "please inspect this rigorously to look for potential issues..."
What's more useful is to have it attempt to not only find such bugs but prove them with a regression test. In Rust, for concurrency tests write e.g. Shuttle or Loom tests, etc.
Re: Claude Code Found a Linux Vulnerability Hidden for 23 Years
#257Earlier quoted context omitted.
> Pasting a big batch of new code and asking Claude "what have I forgotten? Where are the bugs?" It's actually the main way I use CC/codex.
I find Codex sufficiently better for it that I’ve taught Claude how to shell out to it for code reviews
I think OpenAI has actually released an official version of exactly this: https://community.openai.com/t/introducing-codex-plugin-for-...
https://github.com/openai/codex-plugin-cc
I actually work the other way around. I have codex write "packets" to give to claude to write. I have Claude write the code. Then have Codex review it and find all the problems (there's usually lots of them).
Only because this month I have the $100 Claude Code and the $20 Codex. I did not renew Anthropic though.
Re: Claude Code Found a Linux Vulnerability Hidden for 23 Years
#258Earlier quoted context omitted.
Do you not run into too many false positives around "ah, this thing you used here is known to be tricky, the issue is..." I've seen that when prompting it to look for concurrency issues vs saying something more like "please inspect this rigorously to look for potential issues..."
What's more useful is to have it attempt to not only find such bugs but prove them with a regression test. In Rust, for concurrency tests write e.g. Shuttle or Loom tests, etc.
It would be generally good if most code made setting up such tests as easy as possible, but in most corporate codebases this second step is gonna require a huge amount of refactoring or boilerplate crap to get the things interacting in the test env in an accurate, well-controlled way. You can quickly end up fighting to understand "is the bug not actually there, or is the attempt to repro it not working correctly?"
(Which isn't to say don't do it: I think this is a huge benefit you can gain from being able to refactor more quickly. Just to say that you're gonna short-term give yourself a lot more homework to make sure you don't fix things that aren't bugs, or break other things in your quest to make them more provable/testable.)
Re: Claude Code Found a Linux Vulnerability Hidden for 23 Years
#259Pasting a big batch of new code and asking Claude "what have I forgotten? Where are the bugs?" is a very persuasive on-ramp for developers new to AI. It spots threading & distributed system bugs that would have taken hours to uncover before, and where there isn't any other easy tooling. I bet there's loads of cryptocurrency implementations being pored over right now - actual money on the table.
I like biasing it towards the fact that there is a bug, so it can't just say "no bugs! all good!" without looking into it very hard. Usually I ask something like this: "This code has a bug. Can you find it?" Sometimes I also tell it that "the bug is non-obvious" Which I've anecdotally found to have a higher rate of success than just asking for a spot check
As a meta activity, I like to run different codebases through the same bug-hunt prompt and compare the number found as a barometer of quality.
I was very impressed when the top three AIs all failed to find anything other than minor stylistic nitpicks in a huge blob of what to me looked like “spaghetti code” in LLVM.
Meanwhile at $dayjob the AI reviews all start with “This looks like someone’s failed attempt at…”
Re: Claude Code Found a Linux Vulnerability Hidden for 23 Years
#260Earlier quoted context omitted.
A good chunk of the reports are false positives (slop) per the researcher's own admission in his talk. I have no issue sharing the bug reports either; the bugs are better fixed. What I take issue with is that they have basically released the weapon first without thinking about the consequences. And again, if you watch the talk, you'll see how he literally calls others to action to fix the problem. They made a problem…
The “weapon” here is identifying vulnerabilities that were already present and exploitable by malicious actors?
A very shallow dismissal of my point. Is there no room for depth in your logical analysis?
First of all, we don't know whether this particular bug was already being exploited in the wild. We do know that there is a community of experts looking at the Linux kernel and reporting bugs. Yet this bug had never been reported until now. So either nobody ever looked there (unlikely), or they did and didn't find it. Conversely, the LLM found it with a prompt that even a 5-year old can type. That significantly lowers the effort for the attacker, so much that it changes the game. It is, to use a crude analogy, like deploying firearms in a field traditionally fought with sword and shield. So yes, that's the weapon, and these guys released the stuff to the public with no oversight. That should get some people thinking.