Live data from Hacker News

The Ptrace Anti-RE Trick

hkopp.github.io

11–20 of 36 posts

Re: The Ptrace Anti-RE Trick

#11

We used to place an INT3 vectored exception handler on function entry points and do everything interesting inside the exception handler. This made the execution stack basically invisible to every debugger since it doesn't debug exception handlers. You can enable/disable interrupts and tracing and whatever you need to do inside the exception handler to guarantee that nobody can see what you are doing and/or verify tha…

How is the VEH target not immediately visible during static analysis?

Re: The Ptrace Anti-RE Trick

#12
This "self-debugging" technique is common in the Windows world too:

http://profile.maff1t.com/AntiDebugging/

Interestingly enough, VirtualBox does this too, and they call it "hardening", but IMHO it's quite an unexpected and hostile behaviour which is more characteristic of malware.

Re: The Ptrace Anti-RE Trick

#13
In an old $DAY_JOB I used a variant of this as a way to make certain internal errors execute a breakpoint when debugged, and generate an error log if not debugged. iirc this did not happen until after an error had already occurred, so it was not very useful as anti-RE.

Re: The Ptrace Anti-RE Trick

#14
post #13

In an old $DAY_JOB I used a variant of this as a way to make certain internal errors execute a breakpoint when debugged, and generate an error log if not debugged. iirc this did not happen until after an error had already occurred, so it was not very useful as anti-RE.

I'm curious what the context is where you can't just set a breakpoint on the logging function.

Re: The Ptrace Anti-RE Trick

#15
post #8

We used to place an INT3 vectored exception handler on function entry points and do everything interesting inside the exception handler. This made the execution stack basically invisible to every debugger since it doesn't debug exception handlers. You can enable/disable interrupts and tracing and whatever you need to do inside the exception handler to guarantee that nobody can see what you are doing and/or verify tha…

Do you mean using this trick in malware?

No he is referring to happyware.

Re: The Ptrace Anti-RE Trick

#16
post #3

Thankfully this is easy to circumvent: have your debugger catch the ptrace syscall and lie about the result. Also, if antivirus programs haven't already added a signature for any programs that do that, they should.

> Thankfully this is easy to circumvent: have your debugger catch the ptrace syscall and lie about the result. Yeah, Apple iTunes did that IIRC, and it was super easy to bypass.

On Apple platforms ptrace supports an additional flag "PT_DENY_ATTACH", which is what iTunes uses. It causes the target process to exit.

Re: The Ptrace Anti-RE Trick

#17
post #3

Earlier quoted context omitted.

> Thankfully this is easy to circumvent: have your debugger catch the ptrace syscall and lie about the result. Yeah, Apple iTunes did that IIRC, and it was super easy to bypass.

On Apple platforms ptrace supports an additional flag "PT_DENY_ATTACH", which is what iTunes uses. It causes the target process to exit.

Yeah, that's it. Also DVD Player, for example. Pretty trivial to work around.

Re: The Ptrace Anti-RE Trick

#18
post #11

We used to place an INT3 vectored exception handler on function entry points and do everything interesting inside the exception handler. This made the execution stack basically invisible to every debugger since it doesn't debug exception handlers. You can enable/disable interrupts and tracing and whatever you need to do inside the exception handler to guarantee that nobody can see what you are doing and/or verify tha…

How is the VEH target not immediately visible during static analysis?

The VEH target is visible if you know where to look. You can use something like Themida to virtualize/obsfucate the VEH if you really need the VEH to be encrypted.

It's actually not so easy to find the VEH because if you are injecting the VEH into a third party process from another process, then not only does the VEH not exist during static analysis of the binary at rest, but its program address changes on each execution. Moreover, the VEH can be encrypted at rest before it is injected into the second process.

Re: The Ptrace Anti-RE Trick

#19
post #8

We used to place an INT3 vectored exception handler on function entry points and do everything interesting inside the exception handler. This made the execution stack basically invisible to every debugger since it doesn't debug exception handlers. You can enable/disable interrupts and tracing and whatever you need to do inside the exception handler to guarantee that nobody can see what you are doing and/or verify tha…

Do you mean using this trick in malware?

Actually our competitor was reverse engineering our LOB app and we used this to protect trade secrets.
Post reply on HN