31–40 of 438 posts
the palantir testimonial on the landing page is funny
REDACTED REDACTED REDACTED REDACTED REDACTED REDACTED and REDACTED REDACTED? REDACTED REDACTED Antithesis REDACTED REDACTED REDACTED REDACTED, REDACTED REDACTED REDACTED REDACTED. REDACTED REDACTED Palantir REDACTED REDACTED REDACTED REDACTED REDACTED REDACTED REDACTED.
:-)
To me this is very reminiscent of time travel debugging tools like the one used for Firefox’s C++ code, rr / Pernosco: https://pernos.co/
Like this: https://docs.gitlab.com/ee/user/application_security/coverag...
It won't tell you whether the software works correctly, it will just tell you if it raises an exception or crashes.
Put a fuzzer on Chrome for example, you won't catch most of the issues it has, though Chrome actually has tons of bugs and issues, but you may find security issues if you devote a big enough budget to run your fuzzer long time enough to cover all the branches.
So it's good in the case where you use "exceptions as tests", where any minor out-of-scope behavior raises an exception and all the cases are pre-planned (a bit like you baked-in runtime checks, and the fuzzer explores them)
Imagine being proud of working for Palantir.
the palantir testimonial on the landing page is funny
I feel like the idea of the legendary "10x" developer has been bastardized to just mean workers who work 15 hours a day 6.5 days a week to get something out the door until they burn out.
But here's your real 10x (or 50x) productivity. People who implement something very few people even considered or understood to be possible, which then gives amazing leverage to deliver working software in a fraction of the time.
I wonder if they are working on a time travel debugger. If it is truly deterministic presumably you could visit any point in time after a record is made and replay it.
Reminds me of the clever hack of playing back TCP dump logs from prod on a test network, but dialed up. Neat. Naturally I’d prefer professional programmers learn the cognitive tools for manageably reasoning about nondeterminism, but they’ve been around over half a century and it hasn’t happened yet. What’s really interesting to me is that the simulation adequately replicates the real network. One of the more popular…
It’s not an either-or here, though. Part of the challenge is you’re not always thinking about all the non-determinisms in your code, and the interconnections between your code and other code (whose behavior you can sometimes only assume) can make that close to impossible. Part of that is the “your model of the network” critique, but also part of that is “your model of how people will use your software” isn’t necessarily correct either.
I've been super interested in this field since finding out about it from the `sled` simulation guide [0] (which outlines how FoundationDB does what they do). Currently bringing a similar kind of testing in to our workplace by writing our services to run on top of `madsim` [1]. This lets us continue writing async/await-style services in tokio but then (in tests) replace them with a deterministic executor that patches…
Are there public examples of what such a re-write looks like?
Also, are you working at a rust shop that's developing this way?
Final Note, TigerBeetle is another product that was written this way.
Sounds a bit like jockey applied to qemu. Very neat indeed. https://www.cs.purdue.edu/homes/xyzhang/spring07/Papers/HPL-...
There's indeed a connection between record/replay and deterministic execution, but there's a difference worth mentioning, too. Both can tell you about the past, but only deterministic execution can tell you about alternate histories. And that's very valuable both for bug search (fuzzing works better) and for debugging (see for example the graphs where we show when a bug became likely to occur, seconds before it actua…
We needed that for fault tolerance, so we could have a hot synced standby. We did have to record all inputs (and outputs for sanity checking) though.
We did also get a good taste of the debugging superpowers you mention in your blog article. We could pull down a trace from a days trading and replay on our own machines, and skip back and forth in time and find the root cause of anything.
It sounds like what you have done is something similar, but with your own (AMD64) virtual machine implementation, making it fully deterministic and replayable, and providing useful and custom hardware impls (networking, clock, etc).
That sounds like a lot of hard but also fun work.
I am missing something though, in that you are not using it just for lockstep sync or deterministic replays, but you are using it for fuzzing. That is, you are altering the replay somehow to find crashes or assertion failures.
Ah, I think perhaps you are running a large number of sims with a different seed (for injecting faults or whatnot) for your VM, and then just recording that seed when something fails.
Simulation is an interesting approach but I am curious if they ever implemented the simulation wrong would it report errors that don't happen on the target platform or fail to find errors that the target platform reports? How wide the gap is will matter... and how many possible platforms and configurations will the hypervisor cover?