Live data from Hacker News

Things I learned while writing an x86 emulator (2023)

timdbg.com

51–60 of 135 posts

Re: Things I learned while writing an x86 emulator (2023)

#51
post #35
post #7

Earlier quoted context omitted.

> I have never seen a messier architecture. Itanium. Pretty much every time I open up the manual, I find a new thing that makes me go "what the hell were you guys thinking!?" without even trying to.

What sorts of projects are you working on that use Itanium?

None, really. I just happened to get a copy of the manual and start idly reading it when my computer got stuck in a very long update-reboot cycle and I couldn't do anything other than read a physical book.

Re: Things I learned while writing an x86 emulator (2023)

#52

Bonus quirk: there's BSF/BSR, for which the Intel SDM states that on zero input, the destination has an undefined value. (AMD documents that the destination is not modified in that case.) And then there's glibc, which happily uses the undocumented fact that the destination is also unmodified on Intel [1]. It took me quite some time to track down the issue in my binary translator. (There's also TZCNT/LZCNT, which is B…

Another bonus quirk, from the 486 and Pentium area..

BSWAP EAX converts from little endian to big endian and vice versa. It was a 32 bit instruction to begin with.

However, we have the 0x66 prefix that switches between 16 and 32 bit mode. If you apply that to BSWAP EAX undefined funky things happen.

On some CPU architectures (Intel vs. AMD) the prefix was just ignored. On others it did something that I call an "inner swap". E.g. in your four bytes that are stored in EAX byte 1 and 2 are swapped.

  0x11223344 became 0x11332244.

Re: Things I learned while writing an x86 emulator (2023)

#53
post #22

Earlier quoted context omitted.

Did people just... do this by hand (in software), transistor by transistor, or was it laid out programmatically in some sense? As in, were segments created algorithmically, then repeated to obtain the desired outcome? CPU design baffles me, especially considering there are 134 BILLION transistors or so in the latest i7 CPU. How does the team even keep track of, work on, or even load the files to WORK on the CPUs?

It's written in an HDL; IIRC both Intel and AMD use verilog. A modern core is on the order of a million or so lines of verilog. Some of that will be hand placed, quite a bit will just be thrown at the synthesizer. Other parts like SRAM blocks will have their cad generated directly from a macro and a description of the block in question.

To further expound on this. ASIC (like AMD CPUs) is a lot like software work. The engineers that create a lot of the digital logic aren't dealing with individual transistors, instead they are saying "give me an accumulator for this section of code" and the HDL provides it. The definition of that module exists elsewhere and is shared throughout the system.

This is how the complexity can be wrangled.

Now, MOST of the work is automated for digital logic. However, we live in an analog world. So, there is (As far as I'm aware) still quite a bit of work for analog engineers to bend the analog reality into digital. In the real world, changing current creates magnetic fields which means you need definitions limiting voltages and defining how close a signal line can be to avoid cross talk. Square waves are hard to come by, so there's effort in timing and voltage bands to make sure you aren't registering a "1" when it should have been a "0".

Several of my professors were intel engineers. From what they told me, the ratios of employment were something like 100 digital engineers to 10 analog engineers to 1 Physicist/materials engineer.

Re: Things I learned while writing an x86 emulator (2023)

#54

> Writing a CPU emulator is, in my opinion, the best way to REALLY understand how a CPU works Hard disagree. The best way is to create a CPU from gate level, like you do on a decent CS course. (I really enjoyed making a cut down ARM from scratch)

I think both are useful, but designing a modern CPU from the gate level is out of reach for most folks, and I think there's a big gap between the sorts of CPUs we designed in college and the sort that run real code. I think creating an emulator of a modern CPU is a somewhat more accessible challenge, while still being very educational even if you only get something partially working.

Re: Things I learned while writing an x86 emulator (2023)

#55

Earlier quoted context omitted.

Well, I think you're both right. It's satisfying as heck to sling 74xx chips together and you get a feel for the electrical side of things and internal tradeoffs. When you get to doing that for the CPU that you want to do meaningful work with, you start to lose interest in that detail. Then the complexities of the behavior and spec become interesting and the emulator approach is more tractable, can cover more types o…

I think trollied is correct actually. I work on a CPU emulator professionally and while it gives you a great understanding of the spec there are lots of details about why the spec is the way it is that are due to how you actually implement the microarchitecture. You only learn that stuff by actually implementing a microarchitecture. Emulators tend not to have many features that you find in real chips, e.g. caches, sp…

I write retro console emulators for fun, so agree with you 100% :)

Re: Things I learned while writing an x86 emulator (2023)

#57

Intel architecture is loaded with historical artifacts. The switch in how segment registers were used as you went from real mode to protected mode was an incredible hardware hack to keep older software working. I blame Intel for why so many folks avoid assembly language. I programmed in assembly for years using TI's 84010 graphics chips and the design was gorgeous -- simple RISC instruction set, flat address space, a…

What's crazy is that depending on how deep you want to go, a lot of the information is not available in documents published by Intel. Fortunately, if it matters for emulators it typically can be/has been reverse engineering.

Wow, is that true? And the documentation is thousands of pages! I can't understand how Intel keep these processors consistent during development. It must be a really, really hard job.

It was a nice period in history when proper RISC was working well, when we could get stuff to run faster, but getting more transistors was expensive. Now we can't get it to run faster but we can get billions of transistors, making stuff more complicated being the way to more performance.

I wonder if we ever again will get a time where simplification is a valid option...

Re: Things I learned while writing an x86 emulator (2023)

#58
post #19

Earlier quoted context omitted.

> I blame Intel for why so many folks avoid assembly language. x86 (the worst assembly of any of the top 50 most popular ISAs by a massive margin) and tricky MIPS branch delay slots trivia questions at university have done more to turn off programmers from learning assembly than anything else and it's not even close. This is one reason I'm hoping that RISC-V kills off x86. It actually has a chance of once again allow…

What do you find particularly problematic about x86 assembly, from a pedagogical standpoint? I've never noticed any glaring issues with it, except for the weird suffixes and sigils if you use AT&T syntax (which I generally avoid).

I suspect the biggest issue is that courses like to talk about how instructions are encoded, and that can be difficult with x86 considering how complex the encoding scheme is. Personally, I don't think x86 is all that bad as long as you look at a small useful subset of instructions and ignore legacy and encoding.

Re: Things I learned while writing an x86 emulator (2023)

#59

Earlier quoted context omitted.

What do you find particularly problematic about x86 assembly, from a pedagogical standpoint? I've never noticed any glaring issues with it, except for the weird suffixes and sigils if you use AT&T syntax (which I generally avoid).

I suspect the biggest issue is that courses like to talk about how instructions are encoded, and that can be difficult with x86 considering how complex the encoding scheme is. Personally, I don't think x86 is all that bad as long as you look at a small useful subset of instructions and ignore legacy and encoding.

True, encoding is one thing that really sets x86 apart. But as you say, the assembly itself doesn't seem that uniquely horrible (at least not since the 32-bit era), which is why I found the sentiment confusing as it was phrased.

Maybe it's the haphazard SIMD instruction set, with every extension adding various subtly-different ways to permute bytes and whatnot? But that would hardly seem like a beginner's issue. The integer multiplication and division instructions can also be a bit wonky to use, but hardly unbearably so.

Re: Things I learned while writing an x86 emulator (2023)

#60
post #57

Earlier quoted context omitted.

What's crazy is that depending on how deep you want to go, a lot of the information is not available in documents published by Intel. Fortunately, if it matters for emulators it typically can be/has been reverse engineering.

Wow, is that true? And the documentation is thousands of pages! I can't understand how Intel keep these processors consistent during development. It must be a really, really hard job. It was a nice period in history when proper RISC was working well, when we could get stuff to run faster, but getting more transistors was expensive. Now we can't get it to run faster but we can get billions of transistors, making stuff…

There's some info here:

http://www.rcollins.org/secrets/ http://www.rcollins.org/ddj/ddj.html

Notably things like undocumented opcodes, processor modes, undocumented registers, undocumented bits within registers, etc. It's not uncommon to release a specific set of documentation to trusted partners only. Back in the day intel called these "gold books".

Post reply on HN