Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What Data Can’t Do

newyorker.com

Post reply on HN
Print view View original

81–90 of 106 posts

Re: What Data Can’t Do

#81

Post by sjburt »

_dps wrote: post #72
kvathupo wrote: post #70

Earlier quoted context omitted.

I don't quite follow: is not what you described a flaw fundamental to all forecasting; that is, the occurrence of a gross outlier? I should clarify that DL doesn't suffer from the same problem the normality condition has on fat-tails: a failure to capture the skew of the distribution.

It's not characteristic of all forecasting, only purely empirical forecasting. Definitionally, the only way to reason about risk that doesn't appear in training data is non-empirical (e.g. a priori assumptions about distributions, or worst cases, or out-of-paradigm tools like refusing to provide predictions for highly non-central inputs). DL is not any better (or worse) than any other purely empirical method at answe…

I think it's wrong to assume that non-empirical methods can be reliably trusted to give better results. Humans are terrible at avoiding bias or evaluating risks, especially for uncommon events.

Re: What Data Can’t Do

#82

Post by bigbillheck »

carlosf wrote: post #4

I am increasingly worried with people applying ML in everything without any rigour. Statical inference generally only works well in very specific conditions: 1 - You know the distribution of the phenomenon under study (or make an explicit assumption and assume the risk of being wrong) 2 - Using (1), you calculate how much data you need so you get an estimation error below x% Even though most ML models are essentially…

> 1 - You know the distribution of the phenomenon under study (or make an explicit assumption and assume the risk of being wrong)

Nonparametric methods say 'hi'.

Re: What Data Can’t Do

#83

Post by mdip »

lifeisstillgood wrote: post #30
mdip wrote: post #13

Kind of love the initial story in the article about 48-hour wait times. I had a stint writing conferencing software for quite some time, and every once in a while we'd come across a customer requirement that had capabilities which were obvious to us developers "would be misused". As a result, we did the "Thinking, Fast and Slow" pre-mortem to help surface other ways that the system could be attacked (along with what…

conferencing as in 'ComicCon' or 'Zoom'? Can you give an example?

Haha, hadn't even thought of that -- Conferencing as in developing bespoke (and some white-label) software for organizations deploying Office Communications Server (and R2), Lync and ultimately Skype for Business (I do a little Teams work these days but I am focused on other areas, presently).

Re: What Data Can’t Do

#84

Post by dumb1224 »

currymj wrote: post #71
carlosf wrote: post #43

Earlier quoted context omitted.

I disagree, every ML model has some implicit statistical assumption, which is often not well understood by practitioners. At minimum you must assume your underlying process is not fat tailed. If it is, then your training/validation/test data might never be enough to make reliable predictions and your model might break constantly in prod. BTW shifting distributions and fat tailed distributions are sort of equivalent,…

I don't disagree with any of that, but I still think a responsible, clear-thinking ML practitioner can avoid having to assume the form of the data-generating process, depending on their application. In some cases if you care about PAC generalization bounds, it's even the case that the bounds do actually hold for all possible distributions.

I think it's more meaningful to have the discussion in a specific problem domain since statistical inference or ML are just tools to better model a problem / phenomenon. The domain (prior) knowledge -- everything else that's not stats / ML, are the keys to build a more robust model. Leave the problem domain out we are left just with pure mathematical theories and the points can only be proved by simulated data.

Re: What Data Can’t Do

#85

Post by cambalache »

carlosf wrote: post #4

I am increasingly worried with people applying ML in everything without any rigour. Statical inference generally only works well in very specific conditions: 1 - You know the distribution of the phenomenon under study (or make an explicit assumption and assume the risk of being wrong) 2 - Using (1), you calculate how much data you need so you get an estimation error below x% Even though most ML models are essentially…

> You know the distribution of the phenomenon under study

If you know the distribution of the phenomenon under study you dont need ML, that is what probability is for.

> or make an explicit assumption and assume the risk of being wrong

No.You have the Bias/Variance tradeoff here.You can make an explicit assumption about your model or not.

> Using (1), you calculate how much data you need so you get an estimation error below x%

This is extremely complicated for anything except the most trivial toy examples, probably not solvable at all and definitely not the way biological intelligent systems (aka some humans) do it.

Re: What Data Can’t Do

#86

Post by dumb1224 »

kvathupo wrote: post #32
carlosf wrote: post #4

I am increasingly worried with people applying ML in everything without any rigour. Statical inference generally only works well in very specific conditions: 1 - You know the distribution of the phenomenon under study (or make an explicit assumption and assume the risk of being wrong) 2 - Using (1), you calculate how much data you need so you get an estimation error below x% Even though most ML models are essentially…

As currymj commented, this isn't accurate for ML, only for classical statistics. In ML (or more specifically deep learning), we make no distribution-based assumptions, other than the fundamental assumption that our training data is "distributed like" our test data. Thus, there aren't issues with fat-tailed distributions since we make no such normality assumptions. Indeed, with the use of autoencoders, we don't assume…

> I suppose you could say statistics is less "empirical" than ML in the sense that it is axiom-based, whether that is a normality assumption of predictions about a regression line or stock prices following a Wiener process. By contrast, ML is less rationalist by simply reflecting data.

I don't think that's true (or maybe I misunderstood?), I guess your comment "simply reflecting data" means fitting data with a very flexible function (curve)? There are very flexible distributions to fit almost any kind of data e.g https://en.wikipedia.org/wiki/Gamma_distribution or with a composition of them, but as a practitioner you still need to interpret the model and check if it does represent the underlying process well. Both statistical inference and ML are getting there using different methods.

Re: What Data Can’t Do

#87

Post by cloogshicer »

gen220 wrote: post #57
Ozzie_osman wrote: post #20

Data is not a substitute for good judgment, for empathy, for proper incentives. The article focuses on governments and bureaucracies but there's no better example than "data-driven" tech companies, as we A/B test our key engagement metrics all the way to soulless products (with, of course, a little machine learning thrown in to juice the metrics). I wrote about this before: https://somehowmanage.com/2020/08/23/data-i…

I've written the same sentence before! This is so cool! pardon the wall of text. Here's my thesis, curious to hear your thoughts. At some time around 2005, when efficient persistence and computation became cheap enough that any old f500-corp could afford to endlessly collect data forever, something happened. Before 2005, if a company needed to make a big corporate decision, there was some data involved in making the…

Thank you for writing this, I enjoyed reading it and largely agree.

Re: What Data Can’t Do

#88

Post by godelski »

_dps wrote: post #63
kvathupo wrote: post #32

Earlier quoted context omitted.

As currymj commented, this isn't accurate for ML, only for classical statistics. In ML (or more specifically deep learning), we make no distribution-based assumptions, other than the fundamental assumption that our training data is "distributed like" our test data. Thus, there aren't issues with fat-tailed distributions since we make no such normality assumptions. Indeed, with the use of autoencoders, we don't assume…

It is absolutely untrue that DL is immune to fat-fail problems, and it is important that no one operate mission critical systems under this assumption. The two fat tail questions one has to engage are: - is it possible that a catastrophic input might be lurking in the wild that would not be present in a typical training set? Even with a 1M instance training set, a one-in-a-million situation will only appear (and affe…

> It is absolutely untrue that DL is immune to fat-fail problems

In fact, working on fat tail problems is currently a hot topic in ML.

Re: What Data Can’t Do

#89

Post by closeparen »

Eridrus wrote: post #39
kazinator wrote: post #23

> doctors would be given a financial incentive to see patients within forty-eight hours. Not measuring that from the first contact that the patient made is simply dishonest. "Call back in three days to make the appointment, so I can claim you were seen within 48 hours, and therefore collect a bonus" amounts to fraud because the transaction for obtaining that appointment has already been initiated. I mean, they could…

It's really hard to design a not game-able metric. The problem here seems to be that doctors are under-provisioned for some reason, and so long wait times are a form of load shedding for the system. Without addressing this core issue, which individual clinics have little control over because they are generally boxed in by regulations over who can administer medical care, except to rush appointments (which they're pro…

Doctors don’t get to bill for idle time. Being less than fully utilized is leaving money on the table. The idea here is presumably to compensate them for leaving gaps in their schedules.

Re: What Data Can’t Do

#90

Post by boilerupnc »

carlosf wrote: post #4

I am increasingly worried with people applying ML in everything without any rigour. Statical inference generally only works well in very specific conditions: 1 - You know the distribution of the phenomenon under study (or make an explicit assumption and assume the risk of being wrong) 2 - Using (1), you calculate how much data you need so you get an estimation error below x% Even though most ML models are essentially…

[Disclosure: I'm an IBMer - not involved with this work]

With regard to exploitation, IBM research has done some interesting work in the form of an open source "Adversarial Robustness Toolbox" [0]. "The open source Adversarial Robustness Toolbox provides tools that enable developers and researchers to evaluate and defend machine learning models and applications against the adversarial threats of evasion, poisoning, extraction, and inference."

It's fascinating to think through how to design the 2nd and 3rd order side-effects using targeted data poisoning to achieve a specific outcome. Interestingly, poisoning could be to force a specific outcome for a one-time gain (e.g. feed data in a way to ultimately trigger an action that elicits some gain/harm) or to alter the outcomes over a longer time horizon (e.g. Teach the bot to behave in a socially unacceptable way)

[0] https://art360.mybluemix.net/

Post reply on HN