x86 Disassembly • Hacker News Board

Re: x86 Disassembly

#11

Post by calibraxis » Sat, Apr 04, 2015, 9:37 PM UTC

Why does `push eax` perform "much faster" than the following?

  sub esp, 4
  mov DWORD PTR SS:[esp], eax

A brief skim of Intel's "Software Developer’s Manual" (particularly ch. 6 on stacks), didn't seem to find an answer.

While hitting the ALU just for `sub` might be an extra step, doesn't hitting RAM make that a drop in the bucket? (`sub` may account for less than 1%?) Or is there some caching going on, so RAM may be updated in the background?

(I'm not an assembly programmer; very ignorant of what's happening.)

Re: x86 Disassembly

#12

Post by kryptiskt » Sat, Apr 04, 2015, 9:44 PM UTC

calibraxis wrote: post #11

Why does `push eax` perform "much faster" than the following? sub esp, 4 mov DWORD PTR SS:[esp], eax A brief skim of Intel's "Software Developer’s Manual" (particularly ch. 6 on stacks), didn't seem to find an answer. While hitting the ALU just for `sub` might be an extra step, doesn't hitting RAM make that a drop in the bucket? (`sub` may account for less than 1%?) Or is there some caching going on, so RAM may be up…

It is quite hard to envision a scenario where the current stack frame wasn't in any cache level.

Re: x86 Disassembly

#13

Post by calibraxis » Sat, Apr 04, 2015, 9:51 PM UTC

kryptiskt wrote: post #12

calibraxis wrote: post #11

Why does `push eax` perform "much faster" than the following? sub esp, 4 mov DWORD PTR SS:[esp], eax A brief skim of Intel's "Software Developer’s Manual" (particularly ch. 6 on stacks), didn't seem to find an answer. While hitting the ALU just for `sub` might be an extra step, doesn't hitting RAM make that a drop in the bucket? (`sub` may account for less than 1%?) Or is there some caching going on, so RAM may be up…

It is quite hard to envision a scenario where the current stack frame wasn't in any cache level.

Ah ok, it has something to do with caching. So the cost of `sub` can be noticeable. Thanks!

Re: x86 Disassembly

#14

Post by SCHiM » Sat, Apr 04, 2015, 10:08 PM UTC

tibiapejagala wrote: post #7

danieljh wrote: post #3

Earlier quoted context omitted.

As someone looking more into x86_64 assembly, instruction encoding, syscalls and ELF files recently, not only the lack of good starting points but also the amount of work required to get into it is a pity. I'm currently using [0] as a helping hand among other resources which is quite good; my maybe not-so-interesting results are at [1]. For example, to get a good overview of instruction encoding you have to 0/ read t…

Seconding this. I've implemented a part of x86 instruction encoding and you either find resources: * comprehensible, but far from complete (some blogs) * complete, but hard to understand and requiring some implicit knowledge (Intel manual or [1]) Rather than disassembler I recommend writing some simple JIT compiler, with [2] as a starting point. You skip some problems this way. [1] http://ref.x86asm.net/ this seems p…

I use that first reference extensively.

But you have to understand that it's just a reference, it doesn't give you the complete picture. It just shows you the important stuff when you already know where to look.

I've written partial disassemblers/assemblers. And that site has been a huge help to me.

My 2 cents:

Start with being able to decode the mov instruction, with all the different possible memory encodings. Once you understand how you parse the memory/addressing scheme of x86 it's suddenly a whole lot easier. And I agree that writing an assembler to start is probably easier, to write a disassembler it has to be complete, but an assembler doesn't have to support all instructions to work.

Re: x86 Disassembly

#15

Post by jonlox » Sat, Apr 04, 2015, 10:26 PM UTC

tibiapejagala wrote: post #7

danieljh wrote: post #3

Earlier quoted context omitted.

As someone looking more into x86_64 assembly, instruction encoding, syscalls and ELF files recently, not only the lack of good starting points but also the amount of work required to get into it is a pity. I'm currently using [0] as a helping hand among other resources which is quite good; my maybe not-so-interesting results are at [1]. For example, to get a good overview of instruction encoding you have to 0/ read t…

Seconding this. I've implemented a part of x86 instruction encoding and you either find resources: * comprehensible, but far from complete (some blogs) * complete, but hard to understand and requiring some implicit knowledge (Intel manual or [1]) Rather than disassembler I recommend writing some simple JIT compiler, with [2] as a starting point. You skip some problems this way. [1] http://ref.x86asm.net/ this seems p…

I've written a pretty complete assembler a few years back. My advice, if you want to truly learn encoding, you need to write an assembler. The reason being, as you're trying to figure out if your assembler is generating the correct instructions you're going to be looking at it in hexdump format for days or weeks. Pretty soon you're going to notice prefixes, and will be able to visually decode instructions just by looking at them in hex bytes. It's really not that hard after a little practice, and knowing ModRM.

I will emphasize, the Intel manual is pretty much all I used. Along with NASM. I looked at NASM source a lot to figure out what they did, but also used NASM to compare generated instructions. The Intel manual is critical. I would go straight to the authoritative source. It's not hard to follow once you understand the terminology and format a bit. Just keep reading it.

edit: Oh, and, the most important thing to ever know: Intel is little-endian! I cannot stress understanding the importance of this enough. Even when you know this, it's very easy to forget it when looking at code.

Re: x86 Disassembly

#16

Post by awhitworth » Sat, Apr 04, 2015, 11:55 PM UTC

It's funny to see this link come up on HN, because I wrote this book (or much of it) years ago as a college student. I was taking some courses on microprocessor architectures and assembly code, and was in the middle of reading Reversing by Eldad Eilam (which had just come out at the time). I had a mantra back then that the best way to learn a subject was to try and teach it, so I wrote this and several other wikibooks as sort of a study aide for my classes. This also explains why the material appears to cover barely a semester's worth of material and why my name ("Whiteknight") doesn't appear in the edit history after graduation in 2008.

Despite the relatively thin and incomplete coverage of the material, I've heard from several people over the years who appreciated the work as a nice introduction to the topic and even once received a job offer because of it (which didn't work out, for a variety of reasons). All things considered, if I had to change anything it would be the title to make it a little more focused. It's not really a book about disassembly so much as it is an introduction to what high-level language features look like when translated into non-optimized x86 assembly code. Find me a short, catchy title that accurately describes that, and you win some kind of prize.

I doubt I'll ever get back to this book either. I haven't worked with this material at all since school, and don't feel like I have up-to-date knowledge of the subject. Unless somebody else wants to jump in and fill it out, it will probably stay the way you see it now.

I'm glad to see that this book is still around and I'm glad that people are benefiting from it in some small way. I know it doesn't cover nearly what would be needed for a real book on the subject (I do still recommend Eilam's Reversing for book-o-philes) but I think it should be a decent stepping stone to pique interest and get people moving on towards more in-depth treatments.

Re: x86 Disassembly

#17

Post by awhitworth » Sun, Apr 05, 2015, 12:06 AM UTC

calibraxis wrote: post #11

Why does `push eax` perform "much faster" than the following? sub esp, 4 mov DWORD PTR SS:[esp], eax A brief skim of Intel's "Software Developer’s Manual" (particularly ch. 6 on stacks), didn't seem to find an answer. While hitting the ALU just for `sub` might be an extra step, doesn't hitting RAM make that a drop in the bucket? (`sub` may account for less than 1%?) Or is there some caching going on, so RAM may be up…

I don't know exactly why, but I have some theories. First, the two instructions you list there aren't parallelizable, because you have a write to ESP and then a read from it, so the first instruction needs to complete first (or, get far enough in the pipeline for the partial result to be usable). This is almost certainly going to cause some stalls.

Beyond that, the best explanations I've seen are that the push/pop instructions are "optimized" in some way. I don't know if that means they are more optimized than just making the inc/read pair on the ESP atomic so it doesn't stall or if there is more to it.

Re: x86 Disassembly

#18

Post by awhitworth » Sun, Apr 05, 2015, 12:08 AM UTC

xvilka wrote: post #6

This book is missing very actively developed reverse engineering framework radare2 [1], which is supporting not only x86/x86_64 but also arm, mips, ppc, avr, arc, 8051, TI tms320c55x family and even more! [1] http://rada.re/

The title of the book is "X86 Disassembly". It was only ever intended to cover x86. x86_64, mips, ppc, etc are all expressly out of scope.

Re: x86 Disassembly

#19

Post by userbinator » Sun, Apr 05, 2015, 12:36 AM UTC

calibraxis wrote: post #11

Why does `push eax` perform "much faster" than the following? sub esp, 4 mov DWORD PTR SS:[esp], eax A brief skim of Intel's "Software Developer’s Manual" (particularly ch. 6 on stacks), didn't seem to find an answer. While hitting the ALU just for `sub` might be an extra step, doesn't hitting RAM make that a drop in the bucket? (`sub` may account for less than 1%?) Or is there some caching going on, so RAM may be up…

It's a lot smaller (1 byte vs 6), which means less space spent in the cache and decoder, reducing cache misses and decode bandwidth. The x86 also has a dedicated "stack engine" since the Pentium M (but not suprisingly, absent in NetBurst), which contains an adder and copy of the stack pointer to handle push/pop operations. This is faster than using the general-purpose ALUs and memory read/write ports, and also frees those up for use by other non-stack instructions. On the other hand, it means reading/writing the stack pointer explicitly between implicit stack operations incurs a little extra latency to get the values between the stack engine and "real" ESP register synchronised.

Memory reads/writes do take a few more cycles to complete, but since this is a write, the CPU can continue on with other non-dependent instructions following it. All the above information assumes a CPU based on P6 and its successors (Core, Nehalem, Sandy Bridge, Ivy Bridge, Haswell, etc.); NetBurst and Atom are very different.

Linus also has some interesting things to say about using the dedicated stack instructions: http://yarchive.net/comp/linux/pop_instruction_speed.html

Somewhat amusingly, GCC was well known to generate the explicit sub/mov instructions by default, while most other x86 C compilers I knew of, including MSVC and ICC, would always use push.

Re: x86 Disassembly

#20

Post by userbinator » Sun, Apr 05, 2015, 12:53 AM UTC

tptacek wrote: post #2

Write a disassembler at least once. It's much easier than you think it is (even with X86) --- it's essentially a file format parser --- and very illuminating.

It's also much easier to decode x86 instructions when you look at them in octal instead of the hexadecimal that most tables use, since both the main opcode map and ModRM/SIB are organised in a 2-3-3 layout:

http://reocities.com/SiliconValley/heights/7052/opcode.txt

The 8080/8085/Z80 instruction sets also look much better in octal:

http://www.z80.info/decoding.htm