Live data from Hacker News

The unexpected effectiveness of one-shot decompilation with Claude

blog.chrislewis.au

21–30 of 133 posts

Re: The unexpected effectiveness of one-shot decompilation with Claude

#21
post #16

Earlier quoted context omitted.

I've used LLMs to help with decompilation since the original release of GPT-4. They're excellent at recognizing the purpose of functions and refactoring IDA or Ghidra pseudo-C into readable code.

How does it do on things that were originally written in assembly?

This is typically easier because the code was written for humans already.

Re: The unexpected effectiveness of one-shot decompilation with Claude

#22
post #7

If you aren't using LLMs for your reverse engineering tasks, you're missing out, big time. Claude kicks ass. It's good at cleaning up decompiled code, at figuring out what functions do, at uncovering weird assembly tricks and more.

Makes sense because LLMs are quite good at translating between natural languages. Anyway, we're reaching the point where documentation can be generated by LLMs and this is great news for developers.

Documentation is one place where humans should have input. If an LLM can generate documentation, why would I want you to generate it when I can do so myself (probably with a better, newer model)?

Re: The unexpected effectiveness of one-shot decompilation with Claude

#23

If you aren't using LLMs for your reverse engineering tasks, you're missing out, big time. Claude kicks ass. It's good at cleaning up decompiled code, at figuring out what functions do, at uncovering weird assembly tricks and more.

I've been using Claude for months with Ghidra. It is simply amazing.

Re: The unexpected effectiveness of one-shot decompilation with Claude

#24
It's worth noting here that the author came up with a handful of good heuristics to guide Claude and a very specific goal, and the LLM did a good job given those constraints. Most seasoned reverse engineers I know have found similar wins with those in place.

What LLMs are (still?) not good at is one-shot reverse engineering for understanding by a non-expert. If that's your goal, don't blindly use an LLM. People already know that you getting an LLM to write prose or code is bad, but it's worth remembering that doing this for decompilation is even harder :)

Re: The unexpected effectiveness of one-shot decompilation with Claude

#26

It's worth noting here that the author came up with a handful of good heuristics to guide Claude and a very specific goal, and the LLM did a good job given those constraints. Most seasoned reverse engineers I know have found similar wins with those in place. What LLMs are (still?) not good at is one-shot reverse engineering for understanding by a non-expert. If that's your goal, don't blindly use an LLM. People alrea…

Are they not performing well because they are trained to be more generic, or is the task too complex? It seems like a cheap problem to fine-tune.

Re: The unexpected effectiveness of one-shot decompilation with Claude

#27
post #9

Earlier quoted context omitted.

For his defense, it is missing a "Tell HN"

And it isn't always obvious when the commenter is the submitter (no [S] tag like you see on other sites).

whoops, I did indeed miss that this was OP

Re: The unexpected effectiveness of one-shot decompilation with Claude

#29
post #7

Earlier quoted context omitted.

Makes sense because LLMs are quite good at translating between natural languages. Anyway, we're reaching the point where documentation can be generated by LLMs and this is great news for developers.

Maybe documentation meant for other llms to ingest. Their documentation is like their code, it might work, but I don't want to have to be the one to read it. Although of course if you don't vibe document but instead just use them as a tool, with significant human input, then yes go ahead.

Although with code it's implementing functions that don't exist yet and with documentation, it's describing functions that don't exist yet.
Post reply on HN