Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What Data Can’t Do

newyorker.com

Post reply on HN
Print view View original

11–20 of 106 posts

Re: What Data Can’t Do

#11

Post by astrophysician »

carlosf wrote: post #4

I am increasingly worried with people applying ML in everything without any rigour. Statical inference generally only works well in very specific conditions: 1 - You know the distribution of the phenomenon under study (or make an explicit assumption and assume the risk of being wrong) 2 - Using (1), you calculate how much data you need so you get an estimation error below x% Even though most ML models are essentially…

I agree -- as ML becomes increasingly easy to be applied by non-experts or people without a heavy math/stats background, I've seen an increasing volume of arguments against the data science profession (someone the other day called DS the "gate-keepers") but: there be dragons.

Anyone can use SOTA deep learning models today, but in my experience, it's more important to understand the answer to "what are the shortcomings/consequences of using a particular method to solve this problem?" "what is (or could be) biases in this dataset?", etc. It requires a non-trivial understanding of the underlying methodology and statistics to reliably answer these questions (or at least worry about them).

Can you apply deep reinforcement learning to your problem? Maybe. Should you? Well, it depends, and you should understand the pros and cons, which requires more than just the knowledge of how to make API calls. There are consequences to misusing ML/AI, and they may not even be obvious from offline testing and cross validation.

Re: What Data Can’t Do

#13

Post by mdip »

Kind of love the initial story in the article about 48-hour wait times.

I had a stint writing conferencing software for quite some time, and every once in a while we'd come across a customer requirement that had capabilities which were obvious to us developers "would be misused". As a result, we did the "Thinking, Fast and Slow" pre-mortem to help surface other ways that the system could be attacked (along with what we would do to prevent it and how it impacted the original feature).

If you create something, and open it to the public, and there's any way for someone to misuse it for financial incentive (especially if they can do so without consequence), it will be misused. In fact, depending on the incentive, you may find that the misuse becomes the only way that the service is used.

Re: What Data Can’t Do

#14

Post by analog31 »

iagovar wrote: post #10
carlosf wrote: post #4

I am increasingly worried with people applying ML in everything without any rigour. Statical inference generally only works well in very specific conditions: 1 - You know the distribution of the phenomenon under study (or make an explicit assumption and assume the risk of being wrong) 2 - Using (1), you calculate how much data you need so you get an estimation error below x% Even though most ML models are essentially…

ML looks (for many peole) like a way to circunvent your grumpy statiscian saying that the underlying data is worthless and/or you should focus on getting the data pipeline done properly for a logit model on your churn rate.

"Scientist free science," -- being able to optimize systems without understanding them, has been a dream of the business world since the dawn of time. There's always been a market for cookbook recipes that automate the collection of data, and interpretation of results. Before ML, there were "design of experiments," and "statistical quality control."

Re: What Data Can’t Do

#15

Post by polytely »

wodenokoto wrote: post #2

The article seems to stop pretty early, as if something is missing. It’s an anecdote about a government incentive to have doctors see patients within 48 hours causing doctors to refuse scheduling patients later than 48 hours in order to get the incentive bonus. This is not an example of limits of data, but an example of perverse incentives.

Did you use reader mode? because I noticed that when I use firefox's reader mode it will cut off part of the article on the New Yorker site.

Re: What Data Can’t Do

#16

Post by jabroni_salad »

wodenokoto wrote: post #2

The article seems to stop pretty early, as if something is missing. It’s an anecdote about a government incentive to have doctors see patients within 48 hours causing doctors to refuse scheduling patients later than 48 hours in order to get the incentive bonus. This is not an example of limits of data, but an example of perverse incentives.

When a measure becomes a target, it ceases to be a good measure.

This case could be said to be creating misleading data. If the doctor's offices aren't recording appointments more than 48 hours in advance, the System is losing visibility on the total number of people who want appointments. Every office will appear to be 100% efficient even though there is effectively still an invisible waiting list.

Re: What Data Can’t Do

#17

Post by lmm »

Does the use of statistics actually amplify misunderstanding, or merely reveal misunderstandings that were already there? In any of these examples given - predicting rearrests, infant mortality, or so on - it's hard to imagine that someone not using numbers would have reached a conclusion that was any closer to the truth.

Data has its limits, but the solution is usually - maybe even always - more data, not less.

Re: What Data Can’t Do

#18

Post by visarga »

Data is very limited, indeed. We can't predict outside the distribution, or unrelated events (without a causal link), or random events in the future. We should be humble about the limits of data.

Re: What Data Can’t Do

#19

Post by carlmr »

analog31 wrote: post #14
iagovar wrote: post #10

Earlier quoted context omitted.

ML looks (for many peole) like a way to circunvent your grumpy statiscian saying that the underlying data is worthless and/or you should focus on getting the data pipeline done properly for a logit model on your churn rate.

"Scientist free science," -- being able to optimize systems without understanding them, has been a dream of the business world since the dawn of time. There's always been a market for cookbook recipes that automate the collection of data, and interpretation of results. Before ML, there were "design of experiments," and "statistical quality control."

>Before ML, there were "design of experiments," and "statistical quality control."

Statistical quality control, at least the way I know it, is very useful in finding problems in your process. I'm also not sure how this fits with your premise. It's about optimizing systems by first finding out where to look, and then looking there in detail with expert knowledge, i.e. deep understanding of your system.

Re: What Data Can’t Do

#20

Post by Ozzie_osman »

Data is not a substitute for good judgment, for empathy, for proper incentives.

The article focuses on governments and bureaucracies but there's no better example than "data-driven" tech companies, as we A/B test our key engagement metrics all the way to soulless products (with, of course, a little machine learning thrown in to juice the metrics).

I wrote about this before: https://somehowmanage.com/2020/08/23/data-is-not-a-substitut...

Post reply on HN