I love Mickens' work, and think this is overall a great presentation, but I feel like it misses (or maybe just doesn't fully explore) an important point.
Start with the Internet of Things example. He chalks up the abysmal security record of IoT devices to two factors: it keeps IoT devices cheap, and IoT vendors don't understand history. And there's a lot of truth in both these assertions! But they are both just expressing facets of a deeper, more fundamental reason: IoT devices aren't secure because their customers don't demand security.
This deeper problem completely explains why the two higher-level problems he observes exist. Making your product secure makes it more expensive and slower to come to market than just leaving it wide open, and the IoT vendors know their customers care about cost and availability and don't care about security. So they do the rational (in the homo economicus sense of the term) thing and optimize for things their customers are actually willing to pay for.
The same causality can be observed in the ML world. Mickens asks why people are hooking ML systems whose operation isn't fully understood to important things like financial decisionmaking and criminal justice systems. The answer is that the customers demand it. ML is trendy and buzzworthy, so if you're a vendor of (say) financial systems, and you can find some way to incorporate ML into your offerings with a straight face, now you have an attractive new checkbox on the feature list your salespeople dangle in front of potential customers. And once the effectiveness of having that box checked becomes clear, you kind of have to do it, even if you know it'll be ineffective or even worse, or risk losing business to a competitor with fewer scruples.
All of which is to say that what we see playing out in both these scenarios isn't really the vendors' fault. They are instead classic examples of market failure. People end up buying shoddy products because spotting their shoddiness requires technical expertise they don't have; responsible vendors who try not to make shoddy products lose sales to irresponsible vendors who don't; eventually all the responsible vendors are out of business and the only products available to buy are shoddy ones. There are lessons to learn from this, but they're economic rather than technological.