Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Writing a Self-Mutating x86_64 C Program (2013)

shanetully.com

Post reply on HN
Print view View original

41–50 of 61 posts

Re: Writing a Self-Mutating x86_64 C Program (2013)

#41

Post by kazinator »

SomeHacker44 wrote: post #17
benj111 wrote: post #2

Are there any languages that could actually make use of self modifying code? Machines wouldn't have the same problems reasoning about it as humans would. Or is it a question of compilers not being good enough until processor tech made the optimisation not worth it?

LISP macros are compile time self modifying code. So are many other macro systems.

Lisp macros are not self-modifying code in any shape or form. They calculate new syntax tree fragments from existing syntax tree fragments, often using purely functional techniques (no mutation at all, not even of local variables in the macro).

The compiler internals of pretty much any programming language do similar tree to tree transformations: just not ones that the program itself can specify as part of its code. We don't call C self-modifying because a for(;;) loop structure was changed by the compiler into if/goto with generated labels, and then changed again into assembly code.

A macro that mutates the input source code on which it operates wouldn't pass a code review in any competent Lisp shop.

It can happen. For instance (setf (car '(1 2)) 2) is self-modifying code; it tries to replace the 1 with a 2, and that 1 is embedded in the program code itself, in a literal list. This is undefined behavior according to ANSI CL, very similarly to how "abc"[1]++ (modifying a string literal) is undefined behavior in C. Note that this code isn't implementing a macro.

I've never heard of compiled Lisp object code being mutated.

Lisp programs as such can be self-modifying; a common example of that is updating while running: loading new versions of existing modules, replacing old functions with new ones. That doesn't involve modifying code.

Re: Writing a Self-Mutating x86_64 C Program (2013)

#42

Post by simias »

I've written "self-modifying" (really JITed) code for several architectures, mainly ARM, and I when I had to do it for amd64 I was very much surprised by how straightforward it was.

On ARM you have to be very careful to handle the cache correctly when you write self-modifying code, because when you access memory using a regular load or store it's obviously treated like data and goes through the data cache while the instructions are fetched through the instruction cache. So when you write an opcode you have to be careful to flux it out of the data cache (at least up to the point where the caches unify, typically L2 on ARM) and then invalidate the icache to make sure that you get the opcode back.

On modern x86-64 architectures, which typically have a very advanced cache system, I expected to have to deal with that as well. As this article shows, you don't. You just write whatever and you can execute it straight after. When you think about it it's a rather complicated thing for the hardware to implement. I wonder why they do it that way instead of relying on the software to issue flushes in the (relatively rare) situations where a hazard is possible.

Re: Writing a Self-Mutating x86_64 C Program (2013)

#43

Post by jakobdabo »

simias wrote: post #42

I've written "self-modifying" (really JITed) code for several architectures, mainly ARM, and I when I had to do it for amd64 I was very much surprised by how straightforward it was. On ARM you have to be very careful to handle the cache correctly when you write self-modifying code, because when you access memory using a regular load or store it's obviously treated like data and goes through the data cache while the i…

Modern x86s may be Harvard architecture internally (with separate code and data L1 caches), but they still present themselves to the developer as classic Von Neuman, which is easier to program.

Re: Writing a Self-Mutating x86_64 C Program (2013)

#44

Post by Someone »

benj111 wrote: post #2

Are there any languages that could actually make use of self modifying code? Machines wouldn't have the same problems reasoning about it as humans would. Or is it a question of compilers not being good enough until processor tech made the optimisation not worth it?

Before COUs supported stacks, quite a few compilers produced self-modifying code.

Instead of using a stack, many CPUs stored the address to return to in a specific register. If a function wanted to call other functions and return afterwards, it had to store that return address somewhere. Popular solutions where “directly before the start of the function” and “in the jump instruction at the end of the function”. The former is easier for the compiler writer, the latter leads to faster code (the return becomes a simple unconditional jump. With the other approach, you have to load the return address and then jump to it)

And yes, you can’t have reentrant code or even recursion that way. That’s one reason early COBOL and early FORTRAN didn’t support recursion.

Re: Writing a Self-Mutating x86_64 C Program (2013)

#46

Post by simias »

jakobdabo wrote: post #43
simias wrote: post #42

I've written "self-modifying" (really JITed) code for several architectures, mainly ARM, and I when I had to do it for amd64 I was very much surprised by how straightforward it was. On ARM you have to be very careful to handle the cache correctly when you write self-modifying code, because when you access memory using a regular load or store it's obviously treated like data and goes through the data cache while the i…

Modern x86s may be Harvard architecture internally (with separate code and data L1 caches), but they still present themselves to the developer as classic Von Neuman, which is easier to program.

So did they keep it that way purely for historical reason and back-compat? Few people write self-modifying code (or even code loaders) these days, it seems like a tricky feature to implement in hardware for relatively little gain.

Re: Writing a Self-Mutating x86_64 C Program (2013)

#48

Post by jandrese »

simias wrote: post #46
jakobdabo wrote: post #43

Earlier quoted context omitted.

Modern x86s may be Harvard architecture internally (with separate code and data L1 caches), but they still present themselves to the developer as classic Von Neuman, which is easier to program.

So did they keep it that way purely for historical reason and back-compat? Few people write self-modifying code (or even code loaders) these days, it seems like a tricky feature to implement in hardware for relatively little gain.

Depends if the self-modifying code in question is stuff like kernel drivers for popular hardware on widely used operating systems. The x86 manufacturers do bend over backwards for compatibility. You can run 16 bit code on the latest i9 processor if you like, including MSDOS if your motherboard still supports legacy boot modes. That is a remarkable level of backwards compatibility.

Re: Writing a Self-Mutating x86_64 C Program (2013)

#50

Post by benj111 »

Someone wrote: post #44
benj111 wrote: post #2

Are there any languages that could actually make use of self modifying code? Machines wouldn't have the same problems reasoning about it as humans would. Or is it a question of compilers not being good enough until processor tech made the optimisation not worth it?

Before COUs supported stacks, quite a few compilers produced self-modifying code. Instead of using a stack, many CPUs stored the address to return to in a specific register. If a function wanted to call other functions and return afterwards, it had to store that return address somewhere. Popular solutions where “directly before the start of the function” and “in the jump instruction at the end of the function”. The f…

Thanks for that.

Presumably you could have tail call recursion. Although if this is pre stacks, tail call detection is probably asking a bit much.

Post reply on HN