Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Statically Recompiling NES Games into Native Executables with LLVM and Go

andrewkelley.me

Post reply on HN
Print view View original

71–80 of 103 posts

Re: Statically Recompiling NES Games into Native Executables with LLVM and Go

#71

Post by delroth »

simias wrote: post #31
Pxtl wrote: post #4

... while it's not really useful for the NES, which is so old that emulating it does not strain even the crudest modern processor, I'd be excited to see this technique applied to newer consoles for lightweight mobile processors.

I think it might actually work better for more modern hardware. Less handcrafted ASM tricks, much more regular (compiler generated) machine code. And of course no self-modifying code that would be extremely difficult to recompile correctly. Modern hardware (GPU, sound cards,...) is also very similar to what you find on a PC so it would be more straightforward to port all this code. No messing around with the framebuf…

This is so wrong on multiple levels.

First, self-modifying code is still extremely present on modern consoles, at least on the current generation (PS3/X360/Wii/WiiU). Loading code from external media is basically the same problem as self-modifying code (statically recompiling it is trivially equivalent to solving the halting problem).

Second, modern hardware might be similar, but game consoles SDKs export a lot more features to the developers than PC drivers do through DX/GL. The example I take every time is fetch shaders on the WiiU: these are a kind of shaders supported by AMD R600 GPUs but completely abstracted by DX/GL.

Third, maybe there is no more mid-scanline framebuffer tricks, but you have a ton of other problems with the framebuffer: while a PC assumes separate CPU/GPU memory, on modern consoles the framebuffer (and a few textures) are often stored in memory that is shared and synchronized with both CPU and GPU. This is incredibly hard to emulate because a full GPU->CPU FB transfer induces a lot of latency (several hundreds of us last time I checked). IGPs and APUs make this problem a bit more manageable, but we're still missing the graphics API support for shared FB and shared textures.

Some things are better than older consoles but some other things are also a lot worse. JIT-ing shader bytecode is another problem that I don't think has been tackled yet (except maybe for Xbox emulation - which is still in its infancy and for a console using a very old GPU with no use of stuff like compute shaders).

Re: Statically Recompiling NES Games into Native Executables with LLVM and Go

#72

Post by smackfu »

exch wrote: post #54
tibbon wrote: post #17

This is amazing. Also, this is the Dark Magic of programming that I don't think I'll 100% grok in 20 years, but its good to try! edit: now that I think of it, I really need to keep expanding my knowledge. I'm going to go through this post in my terminal and try to at least make the stuff work, so I can start understanding this process. I've been trying to learn Go and C better anyway. Thanks for providing a ground to…

I've looked at this sort of stuff as utter voodoo for a long time. Then I ran into this book[1], and everything just kind of 'clicked' into place in my head. I can't recommend this book often enough. [1]: http://www.nand2tetris.org/book.php In short: It gives you a hands-on approach in designing and building your own computer and programming language, to end up writing and running your own games on the system. * It s…

Incidentally, a computer engineering curriculum starts a few levels below that, with the physics of the transistors and how circuits actually work.

Re: Statically Recompiling NES Games into Native Executables with LLVM and Go

#73

Post by comex »

delroth wrote: post #70
jmhain wrote: post #43

Earlier quoted context omitted.

Why is static pointless and why is dynamic better? If emulating a newer game console, couldn't you get better performance by running a statically recompiled game, since it doesn't have to do the extra work at runtime? Or better yet, couldn't you cross-recompile a game to run it on a platform that couldn't normally handle emulation of the target platform?

You can't: as long as you have self mutating code (including loading more code in memory from another place), static recompiling is simply not possible. It's not a matter of being "better", it's the only solution. As video game consoles start looking more and more like PCs, self mutating code starts to be less frequent and more easily detectable. On the Nintendo 3DS for example, only the OS can allocate executable pa…

Even so, I wonder if you could get better speed for, say, Dolphin, by pausing to do as much statically as possible (perhaps going all the way through LLVM's optimizations) rather than being in a hurry to do everything dynamically without pausing.

Re: Statically Recompiling NES Games into Native Executables with LLVM and Go

#74

Post by delroth »

comex wrote: post #73
delroth wrote: post #70

Earlier quoted context omitted.

You can't: as long as you have self mutating code (including loading more code in memory from another place), static recompiling is simply not possible. It's not a matter of being "better", it's the only solution. As video game consoles start looking more and more like PCs, self mutating code starts to be less frequent and more easily detectable. On the Nintendo 3DS for example, only the OS can allocate executable pa…

Even so, I wonder if you could get better speed for, say, Dolphin, by pausing to do as much statically as possible (perhaps going all the way through LLVM's optimizations) rather than being in a hurry to do everything dynamically without pausing.

I wanted to experiment with this idea at some point (without pausing - using a background thread that does optimization and patches code with the optimized version when it's done) but our current JITCache is just not fit for this. We perform JIT compilation per-basic block, which does not allow for a lot of optimization compared to per-function (following all direct branches as long as possible). Changing that would've required too much time and I got bored of it :(

Re: Statically Recompiling NES Games into Native Executables with LLVM and Go

#75

Post by acdha »

chadseibert wrote: post #63
acdha wrote: post #37

Earlier quoted context omitted.

See also https://github.com/jsmess/jsmess which is a port of an existing emulator codebase using emscripten

I'm not quite sure what the point of this project is, as MESS is rather slow when built natively [1]. Perhaps some of the lower end systems can be emulated, albeit slowly. [1] http://www.mess.org/faq#it_works_but_it_is_way_too_slow

http://archiveteam.org/index.php?title=Javascript_Mess summarizes why: it makes it as widely available as possible and lowers the barrier to entry considerably.

Re: Statically Recompiling NES Games into Native Executables with LLVM and Go

#76

Post by [deleted] »

smackfu wrote: post #72
exch wrote: post #54

Earlier quoted context omitted.

I've looked at this sort of stuff as utter voodoo for a long time. Then I ran into this book[1], and everything just kind of 'clicked' into place in my head. I can't recommend this book often enough. [1]: http://www.nand2tetris.org/book.php In short: It gives you a hands-on approach in designing and building your own computer and programming language, to end up writing and running your own games on the system. * It s…

Incidentally, a computer engineering curriculum starts a few levels below that, with the physics of the transistors and how circuits actually work.

[deleted]

Re: Statically Recompiling NES Games into Native Executables with LLVM and Go

#77

Post by RegEx »

Please forgive me for the bikeshedding, but I have a quick question as a C novice: Is the equality check in the following necessary?

    if (foo & 0x80 == 0x80) {
I thought if you're checking a bit simply anding would be enough (since everything else would be zeros). In other words, could we just use

    if (foo & 0x80) {
If so, then it seems like this would be the preferred form to avoid the precedence issue presented in the article.

Re: Statically Recompiling NES Games into Native Executables with LLVM and Go

#78

Post by jlgreco »

RegEx wrote: post #77

Please forgive me for the bikeshedding, but I have a quick question as a C novice: Is the equality check in the following necessary? if (foo & 0x80 == 0x80) { I thought if you're checking a bit simply anding would be enough (since everything else would be zeros). In other words, could we just use if (foo & 0x80) { If so, then it seems like this would be the preferred form to avoid the precedence issue presented in th…

I am seeing that as basically defensive coding. Similar to how you may do this:

  switch (mode) {
      case FOO: ... break;
      case BAR: ... break;
  }
Do you need the very last break? Technically no, but including it so that you don't forget it when you need it later can be a good idea.

(Also, I think explicitly using comparison operators in conditionals might be the idiomatic thing to do in Go. Someone correct me here if I'm wrong about that.)

Re: Statically Recompiling NES Games into Native Executables with LLVM and Go

#79

Post by Hytosys »

RegEx wrote: post #77

Please forgive me for the bikeshedding, but I have a quick question as a C novice: Is the equality check in the following necessary? if (foo & 0x80 == 0x80) { I thought if you're checking a bit simply anding would be enough (since everything else would be zeros). In other words, could we just use if (foo & 0x80) { If so, then it seems like this would be the preferred form to avoid the precedence issue presented in th…

0x80 is 0b01010000.

Consider that (foo & 0x80) is (0x16 == 0b00010000) if foo is 0x16.

(This may or may not be the intended functionality one way or the other!)

Re: Statically Recompiling NES Games into Native Executables with LLVM and Go

#80

Post by RegEx »

Hytosys wrote: post #79
RegEx wrote: post #77

Please forgive me for the bikeshedding, but I have a quick question as a C novice: Is the equality check in the following necessary? if (foo & 0x80 == 0x80) { I thought if you're checking a bit simply anding would be enough (since everything else would be zeros). In other words, could we just use if (foo & 0x80) { If so, then it seems like this would be the preferred form to avoid the precedence issue presented in th…

0x80 is 0b01010000. Consider that (foo & 0x80) is (0x16 == 0b00010000) if foo is 0x16. (This may or may not be the intended functionality one way or the other!)

I believe 0x80 is 0b1000000
Post reply on HN