Live data from Hacker News

The unexpected effectiveness of one-shot decompilation with Claude

blog.chrislewis.au

91–100 of 133 posts

Re: The unexpected effectiveness of one-shot decompilation with Claude

#91
post #12

Makes me wonder if decompilation could eventually become so trivial that everything would become de-facto open source.

That runs into copyright issues. As someone who does a reasonable amount of decompilation, I wouldn’t ever use an LLM. It falls too close to mechanical transformation territory which is not protected, fair use.

Obviously others aren’t concerned or don’t live in jurisdictions where that would be an issue.

Re: The unexpected effectiveness of one-shot decompilation with Claude

#93
post #50

Earlier quoted context omitted.

Documentation is one place where humans should have input. If an LLM can generate documentation, why would I want you to generate it when I can do so myself (probably with a better, newer model)?

Because it takes time and effort to write documentation. If people __can__ actually read undocumented code with the help of LLMs, why do you need human-written documentation really?

It doesn't need to be written by a human only, but I think generating it once and distributing it with source code is more efficient. Developers can correct errors in the generated documentation, which then can be used by humans and LLMs.

Re: The unexpected effectiveness of one-shot decompilation with Claude

#94

We're wasting Energy reverse-engineering code, which, by definition, already exists now. Oh god. Have you tried asking them to simply open source the code?

Have you ever tried to get a game developer to open source a game? And a Japanese one at that?

Even if they were willing to (they're not) and if they still have the code (they don't), it will contain proprietary code from Nintendo and you'll never get your hands on that (legally)

Re: The unexpected effectiveness of one-shot decompilation with Claude

#95

There are quite a few comments here on code obfuscation. The hardest form of code obfuscation is called homomorphic computing, which is code transformed to act on encrypted data isomorphically to regular code on regular data. The homomorphic code is hard obfuscated by this transformation. Now create a homomorphic virtual machine, that operates on encrypted code over encrypted data. Very hard to understand. Now add da…

Im a encryption noob. Less than a noob. But something I've been wondering about is how can homomorphic computing be opaque/unencryptable?

If you are able to monitor what happens to encrypted data being processed by an LLM, could you not match that with the same patterns created by unencrypted data?

Real simple example, let's say I have a program that sums numbers. One sends the data to an LLM or w/e unencrypted, the other encrypted.

Wouldn't the same part of the LLM/compute machine "light up" so to speak?

Re: The unexpected effectiveness of one-shot decompilation with Claude

#96
post #45

> The ‘give up after ten attempts’ threshold aims to prevent Claude from wasting tokens when further progress is unlikely. It was only partially successful, as Claude would still sometimes make dozens of attempts. Not what I would have expected from a 'one-shot'. Maybe self-supervised would be a more suitable term?

"one-shot" usually just means, one example and its correct answer was provided in the prompt. See also, "zero-shot" / "few-shot" etc.

The article says that having decompiled some functions helps with decompiling others, so it seems like more than one example could be provided in the context. I think the OP was referring to the fact that only a single prompt created by a human was used. But then it goes off into what appears to be an agentic loop with no hard stopping conditions outside of what the agent decides.

We're essentially trying to map 'traditional' ML terminology to LLMs, it's natural that it'll take some time to get settled. I just thought that one-shot isn't an ideal name for something that might go off into an arbitrarily long loop.

Re: The unexpected effectiveness of one-shot decompilation with Claude

#97
post #23

If you aren't using LLMs for your reverse engineering tasks, you're missing out, big time. Claude kicks ass. It's good at cleaning up decompiled code, at figuring out what functions do, at uncovering weird assembly tricks and more.

I've been using Claude for months with Ghidra. It is simply amazing.

What's your workflow? Are you mainly going after x86 targets? Are you using a plugin?

Re: The unexpected effectiveness of one-shot decompilation with Claude

#98
post #2

I've been experimenting with running Claude in headless mode + a continuous loop to decompile N64 functions and the results have been pretty incredible. (This is despite already using Claude in my decompilation workflow). I hope that others find this similarly useful.

Thanks, this is very cool! I've started to dip my toes into this and it's good to see it has potential.

Re: The unexpected effectiveness of one-shot decompilation with Claude

#99

Yeah, it works great for porting as well. I tried it on the assembler sources of Prince of Persia for Apple ii and went from nothing to basics being playable (with a few bugs but still) on modern Mac with SDL graphics within a day.

That's impressive. Did you convert POP from 6502 to C?

Re: The unexpected effectiveness of one-shot decompilation with Claude

#100

We're wasting Energy reverse-engineering code, which, by definition, already exists now. Oh god. Have you tried asking them to simply open source the code?

Have you ever tried to get a game developer to open source a game? And a Japanese one at that? Even if they were willing to (they're not) and if they still have the code (they don't), it will contain proprietary code from Nintendo and you'll never get your hands on that (legally)

from 1999! Plus they probably don't even have the source anymore! A lot of game companies just never kept it!
Post reply on HN