What Data Can’t Do

When I first opened the article in Firefox, I only saw the first two paragraphs (same as the OP). This was true whether it was in reader mode or not. I opened it in Chrome and saw the whole article.

I just tried opening it in Firefox now (a couple hours later) and I see the whole article. If I switch to reader mode I do see it's truncated about halfway through, but I think that's a separate issue from what the OP was seeing.

I agree that ML tends to put weaker assumptions on the data than classical statistics and that it's a good thing.

However most ML certainly makes distributional assumptions - they are just weaker. When you're learning a huge deep net with an L2 loss on a regression task, you have a parametric conditional gaussian distribution under the hood. It's not because it's overparametrized that there's no distributional assumption. Vanilla autoencoders are also working under a multivariate gaussian setup as well. Most classifiers are trained under a multinomial distribution assumption etc.

And fat-tailed distributions are definitely a thing. It's just less of a concern for the mainstream CV problems on which people apply DL.

I think it's almost worse in tech because it largely works. If the government sets a flawed metric, their real goal of pleasing their constituents has failed and theoretically they either have to fix it or lose political support.

But in tech, if your goal is just to make money, soullessly following data will often get you there, to the detriment of everyone else. Clickbait headlines will get you more views. Full-page popup ads will get you more ad clicks/newsletter subscriptions. Microtransactions will get you more sales. Gambling mechanics will get you more microtransactions.

You can say it's a flawed metric, but I think in the end, most people just actually care more about making money than they do about building a good product.

I've written the same sentence before! This is so cool! pardon the wall of text.

Here's my thesis, curious to hear your thoughts.

At some time around 2005, when efficient persistence and computation became cheap enough that any old f500-corp could afford to endlessly collect data forever, something happened.

Before 2005, if a company needed to make a big corporate decision, there was some data involved in making the decision, but it was obviously riddled with imperfections and aggregations biases.

Before 2005, executives needed to be seasoned by experience, to develop this thing we call "Good Judgement", that allows them to make productive inferences from a paucity of data. The Corporate Hierarchy was a contest for who could make the best inferences.

Post-2005, data collection is ubiquitous. Individuals and companies realized that you don't need to pay people with experience any more, you can simply collect better data, and outsource decision-making to interpretations of this data. The corporate hierarchy now is all about how can gather the "best" data, where "best" means grow the money pile by X% this quarter.

"Good Judgement" used to be expected from the CEO, down to at least 1-3 levels of middle management above the front-line people. Now, it appears (to me) to be mostly a feature of the C-Suite and Boards, and it's disappeared elsewhere. Long-term, high-performing companies seem to have a more diffused sense of good judgement. But these are rare. maybe they always have been?

Anyways, as we agree, this has a tendency to lead in problematic directions. Here's my thesis on "why".

Fundamentally, any "data" is reductive of human experience. It's like a photograph that captures a picture by excluding the rest of the world.

Few people seem to understand this analogy, because they think photographs are the ultimate record of an event. Lawyers understand this analogy. With the right framing, angle, lighting (and of course, with photoshop), you can make a photograph tell any story you want.

It's the same issue with data, arguably worse since we don't have a set of standard statistics. We have no GAAP-equivalent for data science (yet?).

Our predecessors understood that data was unreliable, and compensated for this fact by selecting for "Good Judgement". The modern mega-corps demonstrate that we don't have a good understanding of this today, evidenced by religious "data-driven" doctrine, as you describe.

People will say "hey! at least some data is better than no data!", to which I'll say data is useless and even harmful in lieu of capable interpreters. In 2021, have an abundance of data, but a paucity of people who are capable of critical interpretation thereof.

I don't know if it's a worse situation than we had 20 years ago. But it's definitely a different situation, that requires a new approach. I think people are taking notice of it, so I'm hopeful.

This article is totally gibberish. It's a terrible mixture of many unrelated things. Just because those things all have something to do with data (anything can be presented in numeric form), it does not make their issues are about data.

First, the Tony Blair example is not about data. It is a failure of government planning. It's wrong politics and wrong economy.

The G.D.P. example is laughable. G.D.P. is never intended to be used to compare individual cases. What kind of nonsense is this?

And the IQ example. The results are backed by decades of extensive studies. The author thinks picking a few critics can invalidate the whole field. And look! The white supremacist who gave Asians the highest IQ, what a disgrace to his own ideology.

Many more. I feel it's kind of tactic to produce this kind of article. Just glue a bunch of stuff, throw together with somethings seem to be related, bam, you got an article.