Static Analysis at Scale: An Instagram Story

Interesting article. I’d love to hear more about how other folks are getting on with typing in python these days. What’s the preferred tool for limiting etc? I’ve been playing with Microsoft python language server recently and mypy (via vim coc). I haven’t got a config that works well yet but it shows promise and I’m spurred on by how typescript is working well for us in that setup.

Quantifying the number of bugs sounds a bit too close to "zarro boogs found," though I grant that additional data would be nice.

At Google we've been using pytype.

> Lets say we needed to deprecate a function named ‘fn’ for a better named function called called ‘add’.

wondering why they would deprecate fn instead of renaming everywhere with 'add'. I thought that was one of the main big sells of the monolith.

At a certain scale, it becomes prohibitively difficult to rename everything at once, for a few reasons:

* Hyrum's Law - people do terrible things, and a change that seems safe almost never is. Doesn't matter what people should do, if they can they'll do it. Having good tests and scalable CI is the only guard for this.

* Moving a target is hard. If you try to change a moving target A into B, and you're changing a couple thousand references to A, chances are pretty good that while you're getting your A->B change reviewed, someone will add more code that depends on A, and then you can't submit your A->B change without fixing the new reference. The solution here is you check in B, tell people to use B instead of A (which checks the growth of references to A), make A point at B, maybe as an implementation detail (which causes all references to A to depend on B), and incrementally change indirect dependencies on B through A to be direct dependencies on B.

* Depending on a moving target is annoying. If N people write code depending on A, and a commit goes in changing A->B, then that's N people whose productivity you've hurt. In practice this one isn't really that bad, just something to think about.

* Small changes are more easy to roll forward than large ones. This assumes your change is rolled back, but consider this scenario: your change A->B, unbeknownst to you, tickles some non-deterministic behavior (maybe a race condition) when used in a specific way, and your CI doesn't fail when you submit the change at EOD. You come in tomorrow morning and find out that the change was rolled back, because it caused flakiness for N tests went from 0.1% to 25%. If your change was a small, targeted one, it'll be much easier to trace down the non-determinism and understand how it caused this change; but if it was a big one, not only might it be harder to trace down the non-determinism, there might even be other similar non-deterministic bugs that your change is causing. All of these issues will conspire to make it harder for you to make progress on your large-scale change.

When you're changing 10 references, maybe even 20-30 or so (and for some changes this number can go even higher, e.g. renaming an internal Java package), it definitely makes sense to do an A->B type change in a single commit. (And in this situation, the monolith does come in handy, because you don't have to wait for a version bump to propagate the change.) But at O(1K) LOC, this isn't a super tenable.