Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Writing a Self-Mutating x86_64 C Program (2013)

shanetully.com

Post reply on HN
Print view View original

51–60 of 61 posts

Re: Writing a Self-Mutating x86_64 C Program (2013)

#51

Post by Avery3R »

vbezhenar wrote: post #40
AnanasAttack wrote: post #7

Earlier quoted context omitted.

I've seen points like these made before, but never really understood how. Can you give an example of self-modifying code becoming a security issue?

There's a security feature called W^X [0] (also called DEP in Windows). Basically you can use special mode which prevents memory pages to be writeable and executable at the same time, so self-modifying code is not allowed, but it prevents exploits from modifying memory containing executable code. OpenBSD uses it as well. 0: https://en.wikipedia.org/wiki/W%5EX

That's not what DEP is, but it is dependent on DEP. DEP is just another word for the NX bit, that allows a page to be marked as non-executable. With DEP you can still have a page that is RWX if you set its permissions that way.

Re: Writing a Self-Mutating x86_64 C Program (2013)

#52

Post by darkmighty »

JoachimS wrote: post #25
shakna wrote: post #8

Earlier quoted context omitted.

Any _good_ reason? Probably not. Any use? ... Yes. Obfuscation. It might be a terrible practice, but has spawned entire languages, including some that have been used on occasion by industry, as well as the puzzle-solving community at large. (Say, integrate it into a `compile --release` flag.) Right off the back of obfuscation, DRM. If the code modifies itself, especially in unexpected ways, then breaking it becomes h…

A lot of the later demo effects on the c64 is using so called "speedcode" which basically is code generating code on the fly to achieve things like dynamically copying data areas. In effect intelligent loop unrolling i asm. http://codebase64.org/doku.php?id=base:speedcode

Fascinating! I assume this kind of technique has been obsoleted by branch prediction, or are there imaginable modern applications?

Re: Writing a Self-Mutating x86_64 C Program (2013)

#53

Post by shakna »

darkmighty wrote: post #52
JoachimS wrote: post #25

Earlier quoted context omitted.

A lot of the later demo effects on the c64 is using so called "speedcode" which basically is code generating code on the fly to achieve things like dynamically copying data areas. In effect intelligent loop unrolling i asm. http://codebase64.org/doku.php?id=base:speedcode

Fascinating! I assume this kind of technique has been obsoleted by branch prediction, or are there imaginable modern applications?

One of the reasons we needed extremely aggressive unrolling was the size/lack of a stack, making recursion or anything that might look like it difficult/limited/impossible, due to needing to store a location to jump back to.

Anything you could do to minimise the amount of things needed to store, and their size, wasn't wasted time.

And though it might very ocassionally be similarly necessary on some chips today, it's more likely you'll work in assembly on those.

Re: Writing a Self-Mutating x86_64 C Program (2013)

#54

Post by kazinator »

simias wrote: post #42

I've written "self-modifying" (really JITed) code for several architectures, mainly ARM, and I when I had to do it for amd64 I was very much surprised by how straightforward it was. On ARM you have to be very careful to handle the cache correctly when you write self-modifying code, because when you access memory using a regular load or store it's obviously treated like data and goes through the data cache while the i…

> I expected to have to deal with that as well. As this article shows, you don't.

Because x86 has to be compatible with 80286 and 80386 forever.

If special flushing instructions were suddenly needed for self-modifying code to work right, all that ancient MS-DOS and Windows code would break.

Re: Writing a Self-Mutating x86_64 C Program (2013)

#55

Post by wahern »

bloak wrote: post #13
saagarjha wrote: post #5

Earlier quoted context omitted.

JIT languages don't use self-modifying code, they generate new code at runtime and execute that.

Sometimes they will also make small changes to code that was generated earlier, typically by changing a branch instruction to point somewhere else. Dynamic linkers may do that, too, though glibc doesn't normally do it, as far as I know: it prefers to update a pointer to code: same result without needing memory that is both writable and executable and without having to invalidate the instruction cache.

Since the advent of ROP glibc's approach isn't much more secure, though it is saner. A few years ago OpenBSD added the kbind syscall which provides the linker equivalent of W^X. kbind is a kernel-mediated memcpy operation that restricts the code permitted to write to a memory block--specifically, the last piece of code to write to it, which is invariably the linker.

C++ virtual functions are problematic for the same reasons. In C code I've started to avoid function pointers altogether in favor of switch-based dispatch, limiting an attacker to invoking a small, statically defined set of functions, not any arbitrary code in the address space. If I feel the problem demands heavily polymorphic code I'll pull in a scripting language like Lua.

Re: Writing a Self-Mutating x86_64 C Program (2013)

#56

Post by wahern »

benj111 wrote: post #32
benj111 wrote: post #16

Earlier quoted context omitted.

From my delve into such things, in Linux at least the linker is a binary called before the program runs, so its a separate program potentially modifying the code. Interesting philosophical question whether that's still self modifying though.

I appear to be getting downvoted. Assuming its because you think I'm wrong about the separate program. Look up the manpage for ld.so . If you run the strings program on a dynamically linked program the first thing it spits out should be the path to ld.so If you run that program without arguments, it even gives you a usage message.

The OP's point, though, was that the glibc linker (ld.so) doesn't modify code; it largely limits itself to modifying tables of pointers. For access to shared symbols the compiler emits code that indirects through these tables, such as through the global offset table (GOT). The machine code itself is mostly generated to use relative offsets, so the linker doesn't usually need to change static opcode operands.

It does this because ELF is a newer, more abstract executable format. By contrast Windows and AIX have evolved an older dynamic linking strategy which depends more heavily on the linker patching address constants embedded in the code, presumably because of the better backward compatibility. I'm too young to have had first-hand experience with the details, but I do vaguely remember the Linux transition from a.out to ELF and it seemed rather disruptive (though it was all magic to me).

But the Windows approach isn't rightly self-modifying code, either. It's more like a delayed compilation stage. Self-modifying code implies code that rewrites itself dynamically during runtime. Runtime normally means in the normal course of regular program execution, as opposed to link time. From the perspective of the code, link time is a one- or two-time event--static linking and, optionally, dynamic linking--that initializes the application code prior to its first run.

Re: Writing a Self-Mutating x86_64 C Program (2013)

#57

Post by tyingq »

vbezhenar wrote: post #34
tyingq wrote: post #29

Earlier quoted context omitted.

I think you probably read it in Dr Dobbs: http://www.drdobbs.com/embedded-systems/processor-detection-... Not exactly what you described, but very similar: "Differentiating between 8088s and 8086s is trickier. The easiest way I've found to do it is to modify code that's five bytes ahead of IP. Since the prefetch queue of an 8088 is four bytes and the prefetch queue of an 8086 is six bytes, an instruction five bytes a…

Interesting! Does this effect work on modern CPU? I'd imagine that if CPU modifies memory which was consumed by prefetcher, it should reset and reread everything or something like this.

I don't think modern prefetch is linear/fifo.

Re: Writing a Self-Mutating x86_64 C Program (2013)

#58

Post by Someone »

benj111 wrote: post #50
Someone wrote: post #44

Earlier quoted context omitted.

Before COUs supported stacks, quite a few compilers produced self-modifying code. Instead of using a stack, many CPUs stored the address to return to in a specific register. If a function wanted to call other functions and return afterwards, it had to store that return address somewhere. Popular solutions where “directly before the start of the function” and “in the jump instruction at the end of the function”. The f…

Thanks for that. Presumably you could have tail call recursion. Although if this is pre stacks, tail call detection is probably asking a bit much.

You could, but I’m not sure the idea had been discovered yet. https://en.wikipedia.org/wiki/Tail_call#History gives 1977 as its first mention (Steele’s ”Debunking the 'Expensive Procedure Call' Myth“ https://dspace.mit.edu/handle/1721.1/5753#files-area)

Also, many tail-calling functions eventually have to return.

Re: Writing a Self-Mutating x86_64 C Program (2013)

#59

Post by SomeHacker44 »

kazinator wrote: post #41
SomeHacker44 wrote: post #17

Earlier quoted context omitted.

LISP macros are compile time self modifying code. So are many other macro systems.

Lisp macros are not self-modifying code in any shape or form. They calculate new syntax tree fragments from existing syntax tree fragments, often using purely functional techniques (no mutation at all, not even of local variables in the macro). The compiler internals of pretty much any programming language do similar tree to tree transformations: just not ones that the program itself can specify as part of its code.…

While your use case (creation of new syntax tree in a deterministic fashion) is definitely the most common use case of Common Lisp macros - it is not the only one. Common Lisp macros are literally code executed at compile time on the source AST. It needs not make a deterministic change to that AST, needs not be side-effect free, and needs not use only and solely the context of its input AST to output the final AST.

Doesn't mean I suggest doing this, but please do not sell unhygenic CL macros short. :) In fact, in CL one has to go to some lengths to be careful to make macro outputs "safe."

Re: Writing a Self-Mutating x86_64 C Program (2013)

#60

Post by lispm »

SomeHacker44 wrote: post #59
kazinator wrote: post #41

Earlier quoted context omitted.

Lisp macros are not self-modifying code in any shape or form. They calculate new syntax tree fragments from existing syntax tree fragments, often using purely functional techniques (no mutation at all, not even of local variables in the macro). The compiler internals of pretty much any programming language do similar tree to tree transformations: just not ones that the program itself can specify as part of its code.…

While your use case (creation of new syntax tree in a deterministic fashion) is definitely the most common use case of Common Lisp macros - it is not the only one. Common Lisp macros are literally code executed at compile time on the source AST. It needs not make a deterministic change to that AST, needs not be side-effect free, and needs not use only and solely the context of its input AST to output the final AST. D…

If macros are running at compile time, then we don't think that this particular code is self modifying.

If we run code in a Lisp interpreter, then one can write self-modifying code. It's possible to get visible effects. It's also possible to use it during debugging code.

Post reply on HN