Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

rr – record and replay debugger for C/C++

rr-project.org

Post reply on HN
Print view View original

71–80 of 134 posts

Re: rr – record and replay debugger for C/C++

#71

Post by modeless »

See also https://pernos.co/ which is based on rr but adds a queryable database of the whole program execution, which allows you to do things like this:

> [...] just click on the incorrect value. With full program history Pernosco can immediately explain where this value came from. The value is tracked backwards through events such as memcpys or moves in and out of registers until we reach a point where the value "originated" from, and each step in this process is displayed in the "Dataflow" panel that opens automatically. There is no need to read or understand the code and think about what might have happened, we can simply ask the debugger what did happen.

https://pernos.co/examples/race-condition

Re: rr – record and replay debugger for C/C++

#72

Post by rramadass »

whatsakandr wrote: post #41

I've used rr very sucessfully for reverse engineering a large code base using a break on variable change combined with reverse-continue. Took the time to extract critical logic way down.

That sounds very interesting; Do you have a write-up on this that you are willing to share?

Re: rr – record and replay debugger for C/C++

#73

Post by samatman »

polio wrote: post #58
saagarjha wrote: post #57

Earlier quoted context omitted.

…or you can just waste a few bytes? It's not a big deal.

You can, but that makes the type system worse. Also depending on how these few bytes are used, they can add up and drag down performance.

Copying a bunch of stuff because the borrow checker won't let you share it can drag down performance as well. Yes, I do understand why one might conclude that tradeoff is worth it. But it is a tradeoff.

Re: rr – record and replay debugger for C/C++

#74

Post by Veserv »

coldtea wrote: post #63
asveikau wrote: post #43

Earlier quoted context omitted.

I'm not so sure of this. Most code in a debugger doesn't have much to do with the source language. Line mapping and figuring out values for variables happens via symbol information, eg. DWARF, and compilers for multiple languages can produce that in the same format.

Does that neutral way of debugging work as well as having first-class explicit support for the language, or it's more of a lowest common denominator (kind of like what e.g. language support over LSP can offer and how conveniently, compared to what a native Lisp or Smalltalk environment's language support can offer)?

The latter.

The neutral way of debugging is really debugging the raw machine code of a process. This requires OS integration for your low level manipulation primitives. To add language support, you then need to figure out how to define the semantics of your manipulations in terms of the low-level primitives.

If you have a rich runtime, you can add language-level debugging facilities that can operate at a higher level. However, this requires you to implement portions of a debugger in your runtime. Now you have to maintain a language, runtime, and debugger. It also means that if new debugger techniques are invented, such as time travel debugging, you do not get them for free since you embedded a debugger of your own design. So, like many similar things, it is a trade-off of specialization versus maintenance. The perennial question of use a library, or do it yourself.

Re: rr – record and replay debugger for C/C++

#75

Post by forrestthewoods »

Veserv wrote: post #66
forrestthewoods wrote: post #65

On Windows you can use WinDbg for the same thing. It has better support for debugging multi-threaded issues. https://www.forrestthewoods.com/blog/windbg-time-travelling-...

WinDbg uses a instruction-level emulation time travel implementation, so incurs the 10-20x slowdown associated with that technique. rr uses a replay-record time travel debugging implementation, which can incur far less overhead when done correctly. Last I saw, rr has overhead in the 2x slowdown range and, if I remember correctly, I have seen a different record-replay time travel debugger in the 10% range. 10% is 100x…

> 10% is 100x cheaper than WinDbg

If you’re gonna throw around numbers like this you need to cite an actual tool not “if I remember correctly there exists a unicorn”.

Re: rr – record and replay debugger for C/C++

#76

Post by dzaima »

Veserv wrote: post #66
forrestthewoods wrote: post #65

On Windows you can use WinDbg for the same thing. It has better support for debugging multi-threaded issues. https://www.forrestthewoods.com/blog/windbg-time-travelling-...

WinDbg uses a instruction-level emulation time travel implementation, so incurs the 10-20x slowdown associated with that technique. rr uses a replay-record time travel debugging implementation, which can incur far less overhead when done correctly. Last I saw, rr has overhead in the 2x slowdown range and, if I remember correctly, I have seen a different record-replay time travel debugger in the 10% range. 10% is 100x…

More precisely, rr records just system calls & interrupts, so its overhead is largely proportional to how syscall-heavy and multithreaded your code is. If it's single-threaded and mostly just pure computation, you can easily see ~0% slowdown, with replay at the exact same speed. (a quick simple test shows that 'mmap'+'munmap' of a file is 0.5ms, vs 0.006ms outside rr; but many syscalls are buffered in rr's userspace and thus way faster (e.g. a 'stat' on the same file path is only like 300ns slower than native))

Re: rr – record and replay debugger for C/C++

#78

Post by dima55 »

rramadass wrote: post #72
whatsakandr wrote: post #41

I've used rr very sucessfully for reverse engineering a large code base using a break on variable change combined with reverse-continue. Took the time to extract critical logic way down.

That sounds very interesting; Do you have a write-up on this that you are willing to share?

This is the usual killer feature of something like rr. You debug, look at some variable: `p whatever`. You see that its value is wrong. You want to know where this wrong value came from, so you `watch -l whatever` and `rc`. Bam!

Re: rr – record and replay debugger for C/C++

#79

Post by padenot »

leoc wrote: post #77

Are rr’s problems with Ryzen CPUs now firmly in the past or not?

Yes, I use rr all day every day (to record Firefox executions) on a rather recent Threaripper Pro 7950, and also with Pernosco. The rr wiki on GitHub explains how to make it work. Once the small workaround is in place it works very reliably.

Re: rr – record and replay debugger for C/C++

#80

Post by tialaramex »

saagarjha wrote: post #59
polio wrote: post #58

Earlier quoted context omitted.

You can, but that makes the type system worse. Also depending on how these few bytes are used, they can add up and drag down performance.

No, my point is that it doesn't. If your zero-sized types are big your type system is not any worse: it's just less efficient.

If you're focused on just the theoretical correctness of the type system, go back to my first critique: C++ does not have Empty Types. So immediately a whole class of problems that are just a type system question in Rust are imponderable, you can't even say what you meant in C++
Post reply on HN