Live data from Hacker News

Decompiling a Nintendo 64 game in 84 days

blog.chrislewis.au

171–180 of 191 posts

Re: Decompiling a Nintendo 64 game in 84 days

#171

Earlier quoted context omitted.

When I was in high school, graphing calculators were pretty new. New enough that most teachers had not yet encountered them. I was a pretty nerdy kid even then, and I saw graphing calculator as 'portable computer' in 1993! So, of course I wanted one. When our teachers found about them, they banned the use of graphing calculators, some teachers going so far as to only allowing 'four-bangers' (a basic one) for classes…

The smart reason graphing calculators are banned is because students are meant to be learning a concept. You don't learn how to graph to pass a test you take a test so you're pressured to learn how to graph. Same with AI. Having AI generate a for loop doesn't mean you understand the concept of a for loop. You can argue that it's a waste of time to learn about for loops because an AI can write one for you but that's a…

That's why we use different tools when teaching/learning something vs. doing real work.

Another example is computer algebra systems like Mathematica. It would be self-sabotage to use them on your calculus homework, but they're indispensable to the working mathematician, physicist, or engineer, whose time is wasted doing routine calculations by hand.

Re: Decompiling a Nintendo 64 game in 84 days

#172

It's pretty amazing what you can do when you embrace LLMs, figure out how to build one high-quality rigorous project with them, and then start working on more projects. You become a machine with your workflow, once again limited only by your time/energy, tokens, and your discretion on how to spend it.

matching decompilation is a verifiable target. Its hard to create slop that still matches. The same does not hold up for coding with llms normally.

Matching decompilation is a verifiable target that ensures feature parity but it says nothing about being reasonable non slop decompilation.

Disassembler output can relatively easily match original binary, I doubt anyone would consider it decompilation.

Let's say you add requirement of being higher abstraction language than assembly. You can translate the assembly code 1:1 with simple string substitions to C code which would provably replicate the behavior of program perfectly. I would consider that more of hard coded emulation, opposite of JIT like ahead of time code translation. Just because the intermediate langauge is higher level language than assembly doesn't automatically mean that the program text operates at higher level of abstraction than assembly.

I have seen plenty of decompilation projects where you start reading the code many functions contain nonsensical variable reuse (and name that match the register names), address calculations and memory casts and array calculations, arrays of function pointers, offsets within structures which strongly mirrors the assembly code.

The problem is that you can't draw a hard line how good do the variable/function names need to be so that it can be considered decompilation instead of hard coded emulation. It's hard to draw a line for how much of higher level language features you need to use to be considered decompilation. It's hard to define how logical the control flow needs to be. The line becomes blurrier by system programmers approaching it from the other side. Sometimes an extensively optimized code can contain a lot more manual memory mangling, index and offset calculations, manual unrolling than the language requires. So many C programs with homebrew systems of emulating virtual functions using structs of pointers. In such situations it's hard to tell if decompilation is incomplete or whether program was written that way from start.

Another possibility of junk decompilation (which perfectly compiles to byte identical copy of original executable) is program which depends on exact program layout in memory or undefined aspects of programming language. It would perfectly replicate original behavior on original hardware/compiler, but provide 0 insights in intended program behavior and fail miserably when attempting to modify the code or port it to different hardware/compiler. For example the "decompiled" code might intentionally peform out of bounds array access thus touching completely different variable or depend on fallthrough between functions lacking return statement.

Taking it to the extreme would be something like:

```

char data[] = {0x10, 0x12, 0xab, 0xcf, ...}; // bytes corresponding to majority of original program.

int main(){ ((void*())data)(); // cast the array to function pointer and execute it

} ```

It might behave exactly like original program, but that's junk decompilation.

You could add additional restriction that C program isn't allowed to exploit any undefined or machine specific behavior, but then there is very high chance that even original program wouldn't satisfy it.

Original program might also contain parts of it written in assembly for performance reasons or because that was only way to access certain hardware features. What does it mean for decompilation? Is decompiled code not allowed to use the same techniques. If decompiled code is allowed to have some pieces of assembly what stops it from using it for everything. There is no way of knowing precisely how much assembly the original program used.

Re: Decompiling a Nintendo 64 game in 84 days

#173

Earlier quoted context omitted.

That's a terrible analogy. It's more like using a typewriter where the job is calligraphy. I use LLMs at work all the time, and it truly does feel like somebody else coding for me, and me reviewing, cleaning up, and asking for changes. It's really not my work, nor is it my code, but I'm still fine bringing it in. I also don't use LLMs on my personal code done in my free time because I want my brain to still be fully…

There are lots of ways to have a fully functional brain besides writing code manually…

Yes, and all of them involve exercising your mind regularly. I could let LLMs do my personal projects for me and then try to make up the difference with brain teasers, language learning, etc. I choose to exercise my mind because I still love programming, including writing my own code.

I suspect that a lot of people who are going full hog on LLMs aren't actually making up the difference, and are simply offloading their thinking. I choose to not be one of those. I'll use them at work because they make me faster and most of the code is already not mine. It's a means to an end. My personal projects are for the fun and love of programming, not just to have some software in the end.

Re: Decompiling a Nintendo 64 game in 84 days

#175
post #57

For people who've done matching decomps: what burned the most calendar time for you, toolchain freeze or the last 5% of non-matching functions?

I had no issue with the toolchain, as I happened to work on a game (ZZT) whose toolchain of choice was documented by the author himself (well, he was off by one Turbo Pascal release). Lucky!

What took me the most time was actually the beginning: building a mental model of what assembly patterns map into which Pascal patterns, and finding good areas to start writing code from. In hindsight, I should have probably started by generating inline assembly equivalents of each procedure/function. The ending was relatively smooth sailing in comparison, but it is of course much easier if the compiler you're targeting doesn't do much in the way of optimization passes, so it's not representative of decompilation work for more modern targets.

Re: Decompiling a Nintendo 64 game in 84 days

#176
post #68

Earlier quoted context omitted.

That would be a tough change... An easier one, which is how things used to be, that everything enters public domain IF not renewed after X years (let's say 20). Abandoned or complex IP would just become public domain unless the owners themselves figure things out and demonstrate interest with the renewal process. Even a modest fee would lead to a huge gain without much friction for the true rights holders.

Legal would remember to pay the fee for their abandoned works, just in case, while the rest of the company does nil with them. I'd rather address the issue directly and expire the copyright after X years since it was last published. Then again Disney's old "vault" strategy would counter it, and Nintendo is already toying with it to sell through FOMO. How can it be this hard to make companies keep our digital culture…

It's okay if Legal does that. The idea isn't to prevent companies from owning copyrighted works long-term, it's to prevent the waste of abandoned works. There are thousands, if not millions, of works that could be cleanly reused under my proposal.

I don't disagree with the spirit of your argument. I'm just saying that my proposal would be a lot easier to put in practice than anything that hurts companies. The strength of the proposal is exactly that companies will mostly shrug while creating a very clear legal separation of what's "active copyright" and what's an "abandoned work".

By the way, this would also apply to the GPL (and other copyleft licenses).

Re: Decompiling a Nintendo 64 game in 84 days

#177
post #140

Earlier quoted context omitted.

Who said games were well written? Older games are often very sloppy and buggy, and abuse the specific hardware they're designed for to get extra performance, unique graphical effects, etc. Emulators constantly run into games that don't run because of specific hardware nuances that may or may not have been intentionally used (Ex. relying on exact cycle counts, reading uninitialized memory, changing values while they'r…

Sure. I guess I'm assuming that sloppy and buggy software didn't much become popular. Because it was sloppy and buggy.

Why would you assume that? That's not even true of regular software :D

But also, as long as the game works on the specific hardware it was designed for then that's all that mattered, which was my point. If it reads uninitialized memory or depends on exact cycle counts or some undocumented hardware behavior then that's _fine_ if it still works on the real hardware, plenty of that was even intentionally done to achieve things otherwise impossible on that hardware.

At the same time it makes it almost impossible to verify even trivial changes to a function. Once you start accounting for all the potential state differences the answer to "does this function work the same?" will basically always be 'no' unless the code is identical. The only way to eliminate various kinds of state from being a concern is to analyze where the function is _used_, rather than just the function itself, and that's an entirely different and much harder kind of challenge.

Re: Decompiling a Nintendo 64 game in 84 days

#178
post #140

Earlier quoted context omitted.

Sure. I guess I'm assuming that sloppy and buggy software didn't much become popular. Because it was sloppy and buggy.

Why would you assume that? That's not even true of regular software :D But also, as long as the game works on the specific hardware it was designed for then that's all that mattered, which was my point. If it reads uninitialized memory or depends on exact cycle counts or some undocumented hardware behavior then that's _fine_ if it still works on the real hardware, plenty of that was even intentionally done to achieve…

> Why would you assume that?

Why would I assume that sloppy and buggy software would be less popular?

I'll have to get back to you on that one.

Re: Decompiling a Nintendo 64 game in 84 days

#179
post #166

Earlier quoted context omitted.

My observations suggest that frontier models consistently produce correct, high-quality code for small to medium-sized tasks provided the goals are well-specified and testable. Even when they appear to get distracted, they converge on a solution. So what that means is that you the software engineer need to be doing what you should have been doing all along: start with systems analysis, stepwise refine that into a pro…

ITIL is at version 5 now - the idea of a process, is not exactly forgotten in the industry. But you're working with a stochastic system. Even GPU matrix calculations are nondeterministic. It isn't possible, to the delete the entire source code and get exactly the same software back. And any spec, that is specific and non-ambiguous enough to carefully construct the exact needs of each is... Well, that's code. Not a PB…

I said "functionally", not (as Claude says) "byte-identically".

Re: Decompiling a Nintendo 64 game in 84 days

#180
post #168

Earlier quoted context omitted.

Oh honey, the next few years is going to be brutal for you.

Well, considering I accidentally embarrassed the Anthropic rep at work yesterday by producing better code in an hour, then he did in a day, I'm... Pretty sure I'm going to be safe.

lol, I'm sure.
Post reply on HN