Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We hid backdoors in ~40MB binaries and asked AI + Ghidra to find them

quesma.com

Post reply on HN
Print view View original

81–90 of 108 posts

Re: We hid backdoors in ~40MB binaries and asked AI + Ghidra to find them

#81

Post by godelski »

greazy wrote: post #78

Very nitpicky but because I spend a lot of time plotting data: don't arbitrarily color the bar plots without at least mentioning cut offs. Why 19% is orange and 20% is green is a mystery.

It's a pretty common threshold, like 10% is. Be it the 80/20 "Pareto" rule, it's the value of one finger on one hand, or if you really want you stretch the p-value of 0.05 is 1 in 20 odds but that's definitely a stretch though arbitrary anyways. But 20 is a very human number and very common. It's just a division of 5 rather than 4 (I'm assuming you wouldn't have questioned a cutoff at 25%)

Re: We hid backdoors in ~40MB binaries and asked AI + Ghidra to find them

#82

Post by stared »

Tiberium wrote: post #14

I highly doubt some of those results, GPT 5.2/+codex is incredible for cyber security and CTFs, and 5.3 Codex (not on API yet) even moreso. There is absolutely no way it's below Deepseek or Haiku. Seems like a harness issue, or they tested those models at none/low reasoning?

To be honest, it is also our surprise. I mean, I used GPT 5.2 Codex in Cursor for decompiling an old game and it worked (way better than Claude Code with Opus 4.5). We tested for Opus 4.6, but waiting for public API to test on GPT 5.3 Codex.

At the same time, various task can be different, and now all things that work the best end-to-end are the same as ones that are good for a typical, interactive workflow.

We used Terminus 2 agent, as it is the default used by Harbor (https://harborframework.com/), as we want to be unbiased. Very likely other frameworks will change the result.

Re: We hid backdoors in ~40MB binaries and asked AI + Ghidra to find them

#83

Post by godelski »

folex wrote: post #6

> The executables in our benchmark often have hundreds or thousands of functions — while the backdoors are tiny, often just a dozen lines buried deep within. Finding them requires strategic thinking: identifying critical paths like network parsers or user input handlers and ignoring the noise. Perhaps it would make sense to provide LLMs with some strategy guides written in .md files.

Depends what your research question is, but it's very easy to spoil your experiment.

Let's say you tell it that there might be small backdoors. You've now primed the LLM to search that way (even using "may"). You passed information about the test to test taker!

So we have a new variable! Is the success only due to the hint? How robust is that prompt? Does subtle wording dramatically change output? Does "may", "does", "can", "might" work but "May", "cann", or anything else fail? Have you the promoter unintentionally conveyed something important about the test?

I'm sure you can prompt engineer your way you greater success but by doing so you also greatly expand the complexity of the experiment and consequently make your results far less robust.

Experimental design is incredibly difficult due to all the subtleties. It's a thing most people frequently fail at (including scientists) and even more frequently fool themselves into believing stronger claims than the experiment can yield.

And before anyone says "but humans", yeah, same complexity applies. It's actually why human experimentation is harder than a lot of other things. There's just far more noise in the system.

But could you get success? Certainly. I mean you could tell it exactly where the backdoors are. But that's not useful. So now you got to decide where that line is and certainly others won't agree.

Re: We hid backdoors in ~40MB binaries and asked AI + Ghidra to find them

#84

Post by embedding-shape »

stared wrote: post #76
anamexis wrote: post #73

Earlier quoted context omitted.

Why, though? That would make sense if you were just trying to do a comparative analysis of different agent's ability to use specific tools without context, but if your thesis is: > However, [the approach of using AI agents for malware detection] is not ready for production. Then the methodology does not support that. It's "the approach of using AI agents for malware detection with next to zero documentation or guidan…

You can solve any problem with AI if you give enough hints. The question we asked is if they can solve a problem autonomously, with instructions that would be clear for a reverse engineering specialist. That say, I found these useful for many binary tasks - just not (yet) the end-to-end ones.

> The question we asked is if they can solve a problem autonomously

What level of autonomy though? At one point some human have to fire them off, so already kind of shaky what that means here. What about providing a bunch of manuals in a directory and having "There are manuals in manuals/ you can browse to learn more." included in the prompt, if they get the hint, is that "autonomously"?

Re: We hid backdoors in ~40MB binaries and asked AI + Ghidra to find them

#85

Post by embedding-shape »

radlad wrote: post #49
bitexploder wrote: post #23

Earlier quoted context omitted.

I can tell you how I am seeing agents be used with reasonable results. I will keep this high level. I don't rely on the agents solely. You build agents that augment your capabilities. They can make diagrams for you, give you an attack surface mapping, and dig for you while you do more manual work. As you work on an audit you will often find things of interest in a binary or code base that you want to investigate furt…

> But if you have a reasonable subset of "skills" / "agents" you can deploy for various auditing tasks it can absolutely speed you up some. Are people sharing these somewhere?

I think overall you're better off creating these yourself. The more you add to the overall context, the more chance of the model to screw up somewhere, so you want to give it as little as possible, yet still include everything that is important at that moment.

Using the agent and seeing where it get stuck, then creating a workflow/skill/whatever for how to overcome that issue, will also help you understand what scenarios the agents and models are currently having a hard time with.

You'll also end up with fewer workflows/skills that you understand, so you can help steer things and rewrite things when inevitably you're gonna have to change something.

Re: We hid backdoors in ~40MB binaries and asked AI + Ghidra to find them

#87

Post by decidu0us9034 »

anamexis wrote: post #73
stared wrote: post #69

Earlier quoted context omitted.

No. To give it a fair test, we didn't tinker with model-specific context-engineering. Adding skills, examples, etc is very likely to improve performance. So is any interactive feedback. Our example instruction is here: https://github.com/QuesmaOrg/BinaryAudit/blob/main/tasks/lig...

Why, though? That would make sense if you were just trying to do a comparative analysis of different agent's ability to use specific tools without context, but if your thesis is: > However, [the approach of using AI agents for malware detection] is not ready for production. Then the methodology does not support that. It's "the approach of using AI agents for malware detection with next to zero documentation or guidan…

All the docs are already in its training data, wouldn't that just pollute the context? I think giving a model better/non-free tooling would help as mentioned. binja code mode can be useful but you definitely need to give these models a lot of babysitting and encouragement and their limitations shine with large binaries or functions. But sometimes if you have a lot to go through and just need some starting point to triage, false pos are fine.

Re: We hid backdoors in ~40MB binaries and asked AI + Ghidra to find them

#88

Post by mbh159 »

The methodology debate in this thread is the most important part.

The commenter who says "add obfuscation and success drops to zero" is right but that's also the wrong approach imo. The experiment isn't claiming AI can defeat a competent attacker. It's asking whether AI agents can replicate what a skilled (RE) specialist does on an unobfuscated binary. That's a legitimate, deployable use case (internal audit, code review, legacy binary analysis) even if it doesn't cover adversarial-grade malware.

The more useful framing: what's the right threat model? If you're defending against script kiddies and automated tooling, AI-assisted RE might already be good enough. If you're defending against targeted attacks by people who know you're using AI detection, the bar is much higher and this test doesn't speak to it.

What would actually settle the "ready for production" question: run the same test with the weakest obfuscation that matters in real deployments (import hiding, string encoding), not adversarial-grade obfuscation. That's the boundary condition.

Re: We hid backdoors in ~40MB binaries and asked AI + Ghidra to find them

#89

Post by greazy »

godelski wrote: post #81
greazy wrote: post #78

Very nitpicky but because I spend a lot of time plotting data: don't arbitrarily color the bar plots without at least mentioning cut offs. Why 19% is orange and 20% is green is a mystery.

It's a pretty common threshold, like 10% is. Be it the 80/20 "Pareto" rule, it's the value of one finger on one hand, or if you really want you stretch the p-value of 0.05 is 1 in 20 odds but that's definitely a stretch though arbitrary anyways. But 20 is a very human number and very common. It's just a division of 5 rather than 4 (I'm assuming you wouldn't have questioned a cutoff at 25%)

You've missed my point, it's not the thresholds, it's the categories assigned to the thresholds that need explaining.

Re: We hid backdoors in ~40MB binaries and asked AI + Ghidra to find them

#90

Post by anamexis »

stared wrote: post #76
anamexis wrote: post #73

Earlier quoted context omitted.

Why, though? That would make sense if you were just trying to do a comparative analysis of different agent's ability to use specific tools without context, but if your thesis is: > However, [the approach of using AI agents for malware detection] is not ready for production. Then the methodology does not support that. It's "the approach of using AI agents for malware detection with next to zero documentation or guidan…

You can solve any problem with AI if you give enough hints. The question we asked is if they can solve a problem autonomously, with instructions that would be clear for a reverse engineering specialist. That say, I found these useful for many binary tasks - just not (yet) the end-to-end ones.

"With instructions that would be clear for a reverse engineering specialist" is a big caveat, though. It seems like an artificial restriction to add.

With a longer and more detailed prompt (while still keeping the prompt completely non-specific to a particular type of malware/backdoor), the AI could most likely solve the problem autonomously much better.

Post reply on HN