Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How to Build an Evil Compiler

awelm.com

Post reply on HN
Print view View original

21–30 of 38 posts

Re: How to Build an Evil Compiler

#21

Post by hackthesystem »

cpill wrote: post #14

> 2. Modify LoginWithBackdoor.cpp to also accept the password “backdoor” by doing a find-and-replace that modifies the if-condition checking the password Yeah, how is this going to generalise to _any_ program that gets compiled without introducing bugs in the executable? If you can write a backdoor that can accept _any_ program and compormise its integrity you can solve the haulting problem, can't you? So this only w…

Who said it has to "generalize"? No virus generalizes to hack every program. That doesn't mean viruses aren't dangerous.

Also OSS makes up most of the modern stack, so access to source code is a given. And hand-crafting a backdoor when you have the source code is trivial because you can literally change anything you want with confidence.

Re: How to Build an Evil Compiler

#22

Post by hackthesystem »

jcranmer wrote: post #18

The Thompson attack sounds scary, but it's not a real practical concern. Look, as this article demonstrates, it's not hard to build a backdooring compiler. Even if you want to build it in a more robust way than checking filenames, it's really not difficult: it's (admittedly complex) pattern matching, and quite a lot of optimization in fact boils down to pattern matching. The problem is that the pattern matching you'd…

It's possible to make a compiler backdoor that is "updatable" and therefore a lot less brittle. And yes this does make the backdoor easier to detect since it's now communicating over the network. But such flexibility could really future-proof the backdoor and let it evolve over time as the target language changes.

Re: How to Build an Evil Compiler

#23

Post by awelm »

deckarep wrote: post #16

Is it just me…or does article seem a bit contrived? I was expecting to read this to learn about a really powerful hijacking technique when in reality it’s just a program that manipulates your input program. This is something that could easily occur with scripting languages, backend systems, open source, closed source, etc. Basically any black-box system that takes in some input could pre-manipulate the input yielding…

> This is something that could easily occur with scripting languages, backend systems, open source, closed source, etc. Basically any black-box system that takes in some input could pre-manipulate the input yielding an unknown/unexpected output.

IMO thats why it's a scary attack. It's a really simple idea and there are so many ways to apply it

Re: How to Build an Evil Compiler

#24

Post by awelm »

prettyStandard wrote: post #2

Not going to read the whole article because I should be doing work, but... The Ken Thompson hack is not undefeatable. You can detect it using a cross compilation technique comparing the binary output with a clean complier. I think you have to about 4 compilations to figure out if you're infected, but then you don't know which one is infected and which one isn't. You will need more data points to compare. Disassemblin…

Yeah I discuss this at the end of the post. Will link it here:

The current best known defense is Diverse Double-Compiling (DDC), introduced by David Wheeler in 2009. To briefly summarize DDC uses different compilers of the same language to test the integrity of a chosen compiler. In order to pass this test the attacker must have modified all the selected compilers beforehand to insert backdoors into each other, which is a decent amount of work. DDC is a good idea but it has 2 shortcomings that come to mind. The first is that DDC requires all selected compilers to have reproducible builds, meaning that each compiler always generates the exact same executable given the same source code. Reproducible builds aren’t very common because compilers by default include things like timestamps and unique IDs in their builds. The second shortcoming is that DDC becomes less effective for languages that only have a few compilers. Also DDC can’t even be applied to newer languages like Rust with only one compiler. In summary, DDC isn’t a silver bullet and the Thompson attack is still considered to be an open problem.

Re: How to Build an Evil Compiler

#25

Post by schoen »

hackthesystem wrote: post #22
jcranmer wrote: post #18

The Thompson attack sounds scary, but it's not a real practical concern. Look, as this article demonstrates, it's not hard to build a backdooring compiler. Even if you want to build it in a more robust way than checking filenames, it's really not difficult: it's (admittedly complex) pattern matching, and quite a lot of optimization in fact boils down to pattern matching. The problem is that the pattern matching you'd…

It's possible to make a compiler backdoor that is "updatable" and therefore a lot less brittle. And yes this does make the backdoor easier to detect since it's now communicating over the network. But such flexibility could really future-proof the backdoor and let it evolve over time as the target language changes.

For example, you could also make a compiler compile certain other software incorrectly in order to introduce exploitable vulnerabilities in the binaries. When I was working on convincing people of the importance of reproducible builds, I used to use an example where changing a single bit in the binary could introduce a fencepost error by changing a conditional branch operation into a different conditional branch operation. If the conditional branch related to overwriting memory and incrementing pointers (for example), that could make the resulting binary exploitable even though there was no fencepost error in the original source code.

(My examples on x86 involved changing JGE to JG, or JL to JLE, corresponding to changing >= to >, and Combining this with the trusting trust attack, you could have a self-perpetuating bug in the compiler plus a bugdoor in other software. The pattern match for the other software does not necessarily have to be super-specific in that case.

I would definitely agree that this wouldn't survive that many generations of software evolution without active intervention. It definitely wouldn't survive a change of programming language or target machine architecture, for example.

Re: How to Build an Evil Compiler

#26

Post by awelm »

jcranmer wrote: post #18

The Thompson attack sounds scary, but it's not a real practical concern. Look, as this article demonstrates, it's not hard to build a backdooring compiler. Even if you want to build it in a more robust way than checking filenames, it's really not difficult: it's (admittedly complex) pattern matching, and quite a lot of optimization in fact boils down to pattern matching. The problem is that the pattern matching you'd…

It is brittle but an evil compiler can account for that by attempting to compile the hacked program and falling back to real compilation if it gets a compilation error. It could even try downloading an update and try recompiling, but this introduces other ways it can get caught.

Re: How to Build an Evil Compiler

#27

Post by mindslight »

henning wrote: post #6
prettyStandard wrote: post #2

Not going to read the whole article because I should be doing work, but... The Ken Thompson hack is not undefeatable. You can detect it using a cross compilation technique comparing the binary output with a clean complier. I think you have to about 4 compilations to figure out if you're infected, but then you don't know which one is infected and which one isn't. You will need more data points to compare. Disassemblin…

How do you know the other compiler is "clean"? That just begs the question. Also, different compilers could produce different, equally valid instructions, such as debug vs. release builds. There are malware analysts who are good at finding sophisticated malware in binaries. They could probably locate suspicious code that could be obfuscated malicious code. Other than that, if you can't trust any of the compiler vendo…

> How do you know the other compiler is "clean"?

You assert that the other compiler won't contain the exact same backdoor, not that it contains no backdoor.

> different compilers could produce different, equally valid instructions, such as debug vs. release builds

The binaries you're comparing are output by two instances of the same compiler codebase (each instance created by a different compiler). So as long as that compiler is deterministic, each run should have the exact same output.

Re: How to Build an Evil Compiler

#28

Post by can16358p »

Are there any hardware-level examples similar to this?

I mean some sort of "evil CPU": even if all the software is clean, it will act differently on some inputs (read: some pattern of instructions) to produce output in a way that it's indistinguishable at any user-level "views" without a logic analyzer analyzing all the bits in and out the CPU in physical level as electrical signals which is beyond reach practically, while still performing some evil tasks. A similar idea can be expanded to an "evil RAM" or "evil bus" but you probably get it.

I'm pretty sure there are already backdoors already in many CPUs that we aren't aware of, but I was wondering if this particular type of attack has ever been spotted in the wild.

Re: How to Build an Evil Compiler

#29

Post by deckarep »

josephcsible wrote: post #17
deckarep wrote: post #16

Is it just me…or does article seem a bit contrived? I was expecting to read this to learn about a really powerful hijacking technique when in reality it’s just a program that manipulates your input program. This is something that could easily occur with scripting languages, backend systems, open source, closed source, etc. Basically any black-box system that takes in some input could pre-manipulate the input yielding…

The key point is that if your system has an evil compiler, building your own compiler from known-good source code will just give you another evil compiler, no matter how many times you do it. It creates a bootstrapping problem for the victim that doesn't have easy solutions.

Perhaps another way to say it-using an evil compiler could bootstrap any kind of malicious code in the compiled artifact whether it’s a compiler or not.

Re: How to Build an Evil Compiler

#30

Post by josephcsible »

deckarep wrote: post #29
josephcsible wrote: post #17

Earlier quoted context omitted.

The key point is that if your system has an evil compiler, building your own compiler from known-good source code will just give you another evil compiler, no matter how many times you do it. It creates a bootstrapping problem for the victim that doesn't have easy solutions.

Perhaps another way to say it-using an evil compiler could bootstrap any kind of malicious code in the compiled artifact whether it’s a compiler or not.

No, because if that's all it did, just rebuilding your compiler twice would free you from it.
Post reply on HN