Live data from Hacker News

Show HN: Python Concurrency Testing with Frontrun

github.com

1–2 of 2 posts

Show HN: Python Concurrency Testing with Frontrun

#1
Hi HN, I'm the author of Frontrun, a Python concurrency testing library I built over the past 6 months using Claude Code and Codex. It uses bytecode tracing and various forms of monkeypatching to schedule across threads, async and multiprocessing allowing you to find and deterministically reproduce race conditions or deadlocks.

It can understand Python variable assignments, SQL statements and Redis commands, which can detect races that cross abstraction boundaries, e.g. a deadlock between threading.Lock() in one thread and a SQL row lock in another.

To use it, do `pip install frontrun`[1] then put the following in a pytest case:

  result = frontrun.explore(
      setup=Counter,
      workers=Counter.increment,
      count=2,
  )
  result.assert_holds()
Then use `frontrun pytest path/to/test_counter.py`.

My hope is that this will allow authors of libraries which are not currently threadsafe to weather the freethreading transition more easily or to raise the bar for issue submissions involving race conditions. I've put a lot of work into making races deterministic and making the error trace interpretable, demonstrating why that sequence of events that triggers the race. I'd love any usability feedback or success/failure stories you have, and I'm happy to answer any questions about how it works.

[1] MacOS and Linux only.

Show HN: Python Concurrency Testing with Frontrun
github.com

Re: Show HN: Python Concurrency Testing with Frontrun

#2
A bit more about how it works internally. It uses sys.monitoring (or sys.set_trace for older versions of python), as well as LD_PRELOAD / DYNLIB libc patching to track arbitrary io events. It also monkeypatches basic concurrency primitives like the contents of `threading`. (Given how heavily monkeypatched that all is, I've been running frontrun tests separately from an ordinary test suite. The bundled binary handles all that when used with Pytest.)

It tracks "conflicts" at mutable boundaries, building: 1. A "shadow stack" recording when values are assigned to objects which are referenced from more than one thread. 2. A finer-grained model of SQL and Redis items that take into account some of the semantics of their concurrency models.

The code executes normally once, to get conflict points, then it uses a version of an algorithm called Dynamic Partial Order Reduction (DPOR) to efficiently search through all non-isomorphic traces looking for failures (either a crash, a statically detected deadlock, or a violation of the property/invariant provided in the test). This was partially inspired by the rust concurrency testing library Loom, though uses monkey-patching instead of customer concurrency primitives.