Live data from Hacker News

Destroying x86_64 instruction decoders with differential fuzzing

blog.trailofbits.com

71–80 of 113 posts

Re: Destroying x86_64 instruction decoders with differential fuzzing

#71
post #39

Earlier quoted context omitted.

The 8086 introduced the abomination of segment registers. That created many software limitations for much of the 80's. Compilers with 64 K limits on array sizes, or code segment sizes, and similar. By comparison the 680x0 on classic Mac was a pleasure to program. A nice large simple flat address space.

segment registers were a cheap MMU before its age. It was the only way to run code without relocation tables at various addresses. Mind you that at this era the whole operating system fitted in 40kB of RAM! My old turbo-pascal 3.0 editor+compiler was something around 37 kB! Just try to write a hello world of that size nowadays! It's pointless to criticize the past based on 10000 times more powerful hardware nowadays,…

> Just try to write a hello world of that size nowadays!

The only reason why hello world binaries are bloated is because compilers for several compiled-to-native languages statically link many standard library functions into the final output executable.

You can write a hello world DOS terminal program[1] for x86, using the DOS syscall 9 (invoked with interrupt 21h)[2]:

        format MZ

        push    cs
        pop     ds
        mov     ah,9
        mov     dx,hello
        int     21h

        mov     ax,4C00h
        int     21h

        hello db 'Hello world!',24h   
This would compile down to a handful of bytes.

Alternatively, if you want to use modern Windows syscalls (instead of legacy DOS syscalls), you can dynamically link to the Windows system libraries, and implement the hello world like so[3]:

       format PE console                            ; Win32 portable executable console format
       entry _start                                 ; _start is the program's entry point

       include 'INCLUDE/WIN32A.INC'                         

       section '.data' data readable writable       ; data definitions

       hello db "Hello World!", 0
       stringformat db "%s", 0ah, 0

       section '.code' code readable executable     ; code

       _start:
               invoke printf, stringformat, hello   ; call printf, defined in msvcrt.dll
               invoke getchar                       ; wait for any key
               invoke ExitProcess, 0                ; exit the process

       section '.imports' import data readable      ; data imports

       library kernel, 'kernel32.dll',\             ; link to kernel32.dll, msvcrt.dll
               msvcrt, 'msvcrt.dll'

       import kernel, \                             ; import ExitProcess from kernel32.dll
              ExitProcess, 'ExitProcess'

       import msvcrt, \                             ; import printf and getchar from msvcrt.dll
              printf, 'printf',\
              getchar, '_fgetchar'
This too would likely be under a kilobyte.

All of this uses fasm (flat assembler)[4][5].

[1] Source: https://board.flatassembler.net/topic.php?t=1736

[2] DOS syscalls: http://spike.scu.edu.au/~barry/interrupts.html

[3] Source: https://en.wikibooks.org/wiki/X86_Assembly/FASM_Syntax#Hello...

[4] https://flatassembler.net/

[5] https://en.wikipedia.org/wiki/FASM

Re: Destroying x86_64 instruction decoders with differential fuzzing

#72
post #29

Earlier quoted context omitted.

> I really wish Itanium had taken off. IMO it is a superior architecture that was simply ahead of it's time. Itanium was an architecture that was designed for "big iron", i.e. fast, powerful computers. It is thus, in my opinion, much harder to "scale down" to, say, mobile devices than x86.

x86 hasn't really proven that it scales down well for mobile devices either.

My friend uses Asus Zenfone which runs on Intel CPU. It works fine. Intel might have failed with economics or marketing, but engineering works.

Re: Destroying x86_64 instruction decoders with differential fuzzing

#73

Earlier quoted context omitted.

> Segmentation is really nice, and should have been carried on, IMO. Are you serious and are you talking about x86 memory segmentation? Honest question. As far as I'm concerned thinking about CS, DS and ES still gives me shivers and I don't remember to ever have met anyone back then who loved segmentation. The only thing I hated more was the bit planes stuff on the graphics card...

The 16-bit segments weren't great because you were forced to jump through all these hoops in order access the full addressable space. 32-bit segments that weren't crippled led to a lot of really interesting applications that weren't able to be replicated with what we have now. Hence why there's this gap of amd64 CPUs where there's no virtualization support in long mode but 32bit OSes could on the same chips.

Interesting, I wasn't aware that you could use them in protected mode and they were 32-bits wide from the 386 on-wards. It also reminded me of something else: I think Tanenbaum discussed the advantages of the segmented memory model in his operating systems book, but I never paid attention because coming from the 16-bit word I immediately dismissed that idea. I might have to reread that chapter...

Re: Destroying x86_64 instruction decoders with differential fuzzing

#74
post #67
post #50

Earlier quoted context omitted.

z80 and 8086 were both garbage, next to the 68000. It's sad x86 survived this far and is still so popular. Hopefully RISC-V will put an end to this hell.

> z80 and 8086 were both garbage, next to the 68000. At that time, Z80 and 8088/8086 targeted very different market segments than the 68000. So, this is a quite unfair comparison.

OK, the 286 was also garbage compared to the 68000.

Re: Destroying x86_64 instruction decoders with differential fuzzing

#75
post #8

Earlier quoted context omitted.

There is value in the fact that the instruction set is an abstraction. CPU's built with more low-level instruction sets became obsolete faster because they couldn't adapt to new features and maintain compatibility as easily.

Sounds like survivorship bias. x86's longevity is due to the amount of money thrown at the problem. You could surely start with a much cleaner instruction set like the M68k and wind up with a same-or-better result after spending billions on multiple projects to invent new ways of ameliorating the complexity of the ISA, some in parallel, over time. Or you can start by eliminating most of the decode complexity and not…

As much as I like the M68k, I don't think it would be easy to extend it to 64 bits. I'm looking at the MOVE instruction, encoded as:

    00ssRRRmmmMMMrrr

    ss = size
    01 = 8 bits
    10 = 32 bits
    11 = 16 bits
    RRR = src register
    mmm = src mode
    MMM = dest mode
    rrr = dest register
The obvious choice of setting the size bits to '00' for 64 bits is out of the question, because that overlaps many instructions (bit manipulations, bounds checking, several specialized move instructions). The whole instruction set is like this---where you would expect 64-bits to specified are a bunch of instructions instead.

Re: Destroying x86_64 instruction decoders with differential fuzzing

#76

Earlier quoted context omitted.

The 16-bit segments weren't great because you were forced to jump through all these hoops in order access the full addressable space. 32-bit segments that weren't crippled led to a lot of really interesting applications that weren't able to be replicated with what we have now. Hence why there's this gap of amd64 CPUs where there's no virtualization support in long mode but 32bit OSes could on the same chips.

Interesting, I wasn't aware that you could use them in protected mode and they were 32-bits wide from the 386 on-wards. It also reminded me of something else: I think Tanenbaum discussed the advantages of the segmented memory model in his operating systems book, but I never paid attention because coming from the 16-bit word I immediately dismissed that idea. I might have to reread that chapter...

It's worse than that, you can set up 32-bit segments in 32-bit mode, then switch back to 16-bit mode and have them do vaguely sensible things ....

This wasn't defined by Intel, but Windows went on to depend on this feature in order to boot (to access PCI)

(I was once involved in an x86 clone project)

Re: Destroying x86_64 instruction decoders with differential fuzzing

#77
post #13
post #6

Earlier quoted context omitted.

See also this old Microsoft Windows 95-era joke: “ 32 bit extensions and a graphical shell for a 16 bit patch to an 8 bit operating system originally coded for a 4 bit microprocessor, written by a 2 bit company, that can't stand 1 bit of competition. ”

> “32 bit extensions and a graphical shell for a 16 bit patch to an 8 bit operating system originally coded for a 4 bit microprocessor, written by a 2 bit company, that can't stand 1 bit of competition.” DOS was a 16 bit operating system. The 8088 (the processor of the IBM-PC) was an 16 bit (if you consider the instruction set) or 8 bit (if you consider the width of the data bus) processor.

to be fair the 8088 was essentially the same die as an 8086 with an 8-bit bus .... memory was expensive back then

Re: Destroying x86_64 instruction decoders with differential fuzzing

#78

Wow, this a wonderfully rich post! I had a question about the following statement: >"In short, it’s a mess, with each generation adding and removing functionality, reusing or overloading instructions and instruction prefixes, and introducing increasingly complicated switching mechanisms between supported modes and privilege boundaries." Can someone elaborate on how a instruction at the machine level can be "overloade…

A very simple example of this is the accumulator form of XCHG (exchange values). The accumulator form (AX/EAX/RAX is the accumulator) is encoded as:

    10010rrr

    rrr = register
    000 = AX/EAX/RAX
    001 = CX/ECX/RCX
    010 = DX/EDX/RDX
    011 = BX/EBX/RBX
    100 = SP/ESP/RSP
    101 = BP/EBP/RBP
    110 = SI/ESI/RSI
    111 = DI/EDI/RDI
More inportantly, the XCHG instruction does NOT affect the flags. So the upshot is that the instruction `XCHG accumulator,accumulator` is effectively a no-operation, and yes, Intel does document the encoding for NOP as being

    10010000
or `XCHG accumulator,accumulator`. Early chips in the x86 line (8086, 80286) actually did the physical exchange, but as the architecture grew with OoO operations, the chips now detect 0x90 as a NOP instruction and deal with it differently (most likely to prevent pipeline stalls waiting for the results of `XCHG accumulator,accumulator').

Re: Destroying x86_64 instruction decoders with differential fuzzing

#79
post #50

>... a 40-year-old 16-bit ISA designed to be source-compatible with a 50-year-old 8-bit ISA. In fairness to the Intel of that era, they actually did a really good job with this. They gained basically zero warts from the 8080 assembler source compatibility. They mostly set out to make the best variable length 16 bit instruction set they could. They had significant competition at the time and they pretty much had to ma…

z80 and 8086 were both garbage, next to the 68000. It's sad x86 survived this far and is still so popular. Hopefully RISC-V will put an end to this hell.

The Z-80 came out in 1976, three years before the 68000 and was an extension of the 8080. I don't see comparing the two as fair or sensible.

Re: Destroying x86_64 instruction decoders with differential fuzzing

#80
post #26

Earlier quoted context omitted.

Saw die shots of Atom? Decoder makes more than half of the core, and there is no instruction cache. In a loop, you will be spending more joules on decoding than actual computations.

What die shots are you looking at? I'm looking at a Silverthorne die and most of the frontend is taken up by the L1I$. https://en.wikichip.org/wiki/intel/microarchitectures/bonnel...

My bad
Post reply on HN