Magic-trace – High-resolution traces of what a process is doing

I think for a lot of people there's a pretty big difference between these two business models:

- We use cash to buy circuit boards, screens, enclosures, etc, write software, and sell mobile phones.

- We use cash to rent a building, order pallets of inventory, and sell that inventory locally to walk-in customers.

- We use cash to buy shares, hold onto them for a bit, and sell those same shares and make money off the spread.

I'm not making any kind of comment at all about the value of market makers, just... those three businesses feel like they're different models.

I’m probably missing something so apologies if this is obvious: does this only work on compiled programs or could it work on any arbitrary running code. Everything from Firefox to my random python script?

Windows (AMD or Intel) has Time Travel Debugging: https://docs.microsoft.com/en-us/windows-hardware/drivers/de..., although this has extremely high performance overhead.

There's also the regular performance traces you can capture with wpr and friends. I don't think these provide function-level traces, and I also don't think it's possible to do that (but I could be wrong). You just get sampled callstacks, which may or may not be enough for your needs.

In my experience on Windows you need to instrument applications to get function-level tracing.

Does that cover the entire financial industry?

One time, I wrote a simple set of tools at my company.

1) A program that inserted a macro invocation with a GUID, at every single new scope. "{", except not switch, struct, or class, etc.

2) I made the macro instantiate an object on the stack, passing in the GUID. In the constructor and in the destructor, it called a singleton with thread-local storage to a file pointer, where it would append a few bytes indicating whether it was entering or exciting scope, what the GUID was, and what the time was. In this way, each thread was writing to its own file.

3) I made a program which walked my source, looking for file name, line number for each of my macro invocations, and the GUID. If I were more sophisticated, I would have tried to get the function name out of it, too.

4) I made a program which would turn one of the thread's files into a Visual Studio recognized output. Basically "filename(linenumber): [content, such as the time]". Then I set that up as a tool in Visual Studio, and when I would run it, it would output in a Visual Studio window. The reason for that was then I could hit (I think) F4 and Shift-F4 to step forward and backward through the output, and each time it would jump to the source code at that location.

So then I had a forward-and-backward time travelling debug script. I think I also started manually passing in function parameters into the macros, which would format (a: "a") on the debug line, too.

We had automated testing of our whole integrated application. I wanted to record my output from each automated test. Then when I was checking in new code, I could see which new GUIDs were never touched by any of our integration tests, to tell me how much coverage we had. And I could tell the testers which automated tests were most likely to exercise my code changes.

I liked that the GUIDs would have been stable, even as code moved. (Unlike file name, line number, or even class and function name.)

And yes, seeing this in the code wasn't great:

{ TIMER("5c7c062f-84a3-40d0-b7cd-77bd9db59f3e");

  // real code

}

I wanted to teach Visual Studio how to basically ignore those, and if I copied code and pasted it, have it generate new GUIDs when I pasted.

But I could imagine using the output to generate the fire charts, and other debugging tools, like in the article.

And it all compiled to 0, in Release mode.

The payoff of this felt large, and the cost felt small. But the biggest pain was that humans would see these macro invocations, and need to maintain them.

So I chickened out and didn't force my coworkers to see all of this.