Micro-libraries should never be used

I was assuming that the constituents of the metapackage aren’t completely independent. TFA is about micro-libraries the size of is-number and left-pad. If you only consider completely independent libraries of that type, you won’t get very far. A more realistic take would be hundreds of such micro-libraries that aren’t self-contained, but instead build on each other to successively provide more powerful functions. And then the more transitive dependencies become more like “internal code” in regular libraries, and you’ll get the same churn, and you’ll be similarly unable to change things due to the interdependencies. I don’t see how you can end up with a comparable functionality a regular library provides without the same potential of running into the issues you note.

> all of those small composable units, that were once developed by random people, were turned into a meta-package at some point

No they weren’t. Every UNIX I used in the 80s and 90s shipped with those little composable building blocks as part of the OS, and GNU bundled them in things like coreutils forever. It’s not like there was some past time when there were independent little things like cat and wc and so on written by random people on the internet that somehow got bundled into a meta-package after they existed. That didn’t happen.

Sometimes importing zombie "undead code" libraries can be beneficial!

I just refactored a bunch of python computer vision code that used detectron2 and yolo (both of which indirectly use OpenCV and PyTorch and lots of other stuff), and in the process of cleaning up unused code, I threw out the old imports of the yolo modules that we weren't using any more.

The yololess refactored code, which really didn't have any changes that should measurably affect the speed, ran a mortifying 10% slower, and I could not for the life of me figure out why!

Benchmarking and comparing each version showed that the yololess version was spending a huge amount of time with multiple threads fighting over locks, which the yoloful code wasn't doing.

But I hadn't changed anything relating to threads or locks in the refactoring -- I had just rearranged a few of the deck chairs on the Titanic and removed the unused yolo import, which seemed like a perfectly safe innocuous thing to do.

Finally after questioning all of my implicit assumptions and running some really fundamental sanity checks and reality tests, I discovered that the 10% slow-down in detectron2 was caused by NOT importing the yolo module that we were not actually using.

So I went over the yolo code I was originally importing line by line, and finally ran across a helpfully commented top-level call to fix an obscure performance problem:

https://github.com/ultralytics/yolov5/blob/master/utils/gene...

    cv2.setNumThreads(0)  # prevent OpenCV from multithreading (incompatible with PyTorch DataLoader)

Even though we weren't actually using yolo, just importing it, executing that one line of code fixed a terrible multithreading performance problem with OpenCV and PyTorch DataLoader fighting behind the scenes over locks, even if you never called yolo itself.

So I copied that magical incantation into my own detectron2 initialization function (not as top level code that got executed on import of course), wrote some triumphantly snarky comments to explain why I was doing that, and the performance problems went away!

The regression wasn't yolo's or detectron2's fault per se, just an obscure invisible interaction of other modules they were both using, but yolo shouldn't have been doing anything globally systemic like that immediately when you import it without actually initializing it.

But then I would have never discovered a simple way to speed up detectron2 by 10%!

So if you're using detectron2 without also importing yolo, make sure you set the number of cv2 threads to zero or you'll be wasting a lot of money.

> You can write isNumber(foo) instead of typeof foo === "number".

Indeed you can, but it depends what isNumber does. This is more like what it should do IMO:

function isNumber( foo ) { return ( (typeof foo === "number") && (foo == foo)) || ((typeof foo === 'object') && (foo instanceof Number) ); }

And that is I think the value of micro libs, at least in JS, you don't want to think about all the edge cases when you only want to check if something is a Number.

This is an argument for having a library that provides that function, but it is not an argument that it should be a micro-library.