What Data Can’t Do • Hacker News Board

Re: What Data Can’t Do

#101

Post by rsfern » Thu, Apr 01, 2021, 2:26 AM UTC

clircle wrote: post #97

sfink wrote: post #50

Earlier quoted context omitted.

Personally, I think the main problem with ML is simpler: it works well for interpolation, and is crap for extrapolation. If the outputs you want are well within the bounds of your training data set, ML can do wonders. If they aren't, it'll tell you that in 20 years everyone will be having -0.2 children and all the other species on the planet will start having to birth human babies just so they can be thrown into the…

Outside of simple time series, I'm not aware of any good way to extrapolate.

One way to extrapolate is to use a mechanistic or semi-mechanistic model. The recent advances in neural differential equations are a really cool example of this

Re: What Data Can’t Do

#102

Post by texasbigdata » Thu, Apr 01, 2021, 3:12 AM UTC

cloogshicer wrote: post #87

gen220 wrote: post #57

Earlier quoted context omitted.

I've written the same sentence before! This is so cool! pardon the wall of text. Here's my thesis, curious to hear your thoughts. At some time around 2005, when efficient persistence and computation became cheap enough that any old f500-corp could afford to endlessly collect data forever, something happened. Before 2005, if a company needed to make a big corporate decision, there was some data involved in making the…

Thank you for writing this, I enjoyed reading it and largely agree.

Same, but the sentance "Individuals and companies realized that you don't need to pay people with experience any more, you can simply collect better data, and outsource decision-making to interpretations of this data." is probably demonstrably false and at a minimum it'd be very difficult to prove its true over a bull hypothesis that were as good as, if not marginally better than before as a species.

To an extreme it's "well we got horse carriages and cars so like no one is ever gonna run a fast marathon ever again, were outsourcing everything to the hay eaters".

Yeah no. And not that we shouldn't perhaps be more contemplative as a society, but as a species we generally don't atrophy capability that would other wise be complentary and beneficial to us.

Case and point, it's relatively well known "management ability" is a thing across cultures, experience, industry and training. The metrics point to at least in a narrow sense data driven executives outperforming their "solely intuitive" bunch.

Re: What Data Can’t Do

#103

Post by [deleted] » Thu, Apr 01, 2021, 3:24 AM UTC

potatoman22 wrote: post #64

fractionalhare wrote: post #58

Earlier quoted context omitted.

> In ML (or more specifically deep learning), we make no distribution-based assumptions, other than the fundamental assumption that our training data is "distributed like" our test data. Okay, so that's about the same as classical statistics. You're just waiving the requirement to know what the distribution is. You are still assuming there exists a distribution and that it holds in the future when you apply the model…

Of course there's a distribution behind the data. The parent commenter was saying not all machine learning techniques need to know that distribution, as a refute to their parent comment.

[deleted]

Re: What Data Can’t Do

#104

Post by kragen » Thu, Apr 01, 2021, 4:11 AM UTC

carlosf wrote: post #92

sjburt wrote: post #81

Earlier quoted context omitted.

I think it's wrong to assume that non-empirical methods can be reliably trusted to give better results. Humans are terrible at avoiding bias or evaluating risks, especially for uncommon events.

Food for thought: if every method for predicting event x is terrible, then you might as well not try to predict x and build your life in such way that you never expose yourself to the risk of x happening.

From a Bayesian point of view, that amounts to a "prediction" that the probability of event x is so significant that you should build your life around it. But I guess if you knew enough for that sentence to make sense you wouldn't have posted your comment. So, suffice it to say that Bayesian decision theory cuts the knot you're talking about.

Re: What Data Can’t Do

#105

Post by mjburgess » Thu, Apr 01, 2021, 8:36 AM UTC

lmm wrote: post #93

mjburgess wrote: post #21

Earlier quoted context omitted.

It's pretty trivial to predict things without "data". Data just means using some measurement system to obtain measurements of some target phenomenon. Many targets cannot be measured, or have not occurred to be measured. Reasoning counter-factually is trivial: What would happen if I dropped this object in this place in which an object, of this kind, has never been dropped before? Well apply relevant models, etc. and "…

But where does that model come from if not from data? We might use some logical principles to inform our model - but don't those principles themselves also ultimately have to be inferred from data about the world?

measurement of our bodies engaged in deliberate action

measurements of the world resolve ambiguities; they do not 'contain' descriptions of the world, not can they provide any

measurements of objects must be interpreted by models

Re: What Data Can’t Do

#106

Post by Eridrus » Thu, Apr 01, 2021, 4:57 PM UTC

closeparen wrote: post #89

Eridrus wrote: post #39

Earlier quoted context omitted.

It's really hard to design a not game-able metric. The problem here seems to be that doctors are under-provisioned for some reason, and so long wait times are a form of load shedding for the system. Without addressing this core issue, which individual clinics have little control over because they are generally boxed in by regulations over who can administer medical care, except to rush appointments (which they're pro…

Doctors don’t get to bill for idle time. Being less than fully utilized is leaving money on the table. The idea here is presumably to compensate them for leaving gaps in their schedules.

What you're describing is effectively what doctors did: they left their entire calendar free until the last moment and only took appointments then.

It turns out this is not actually what people want; people want this availability to exist, but also do not want to be turned away if they book ahead of time, which points to this being a capacity problem, not a scheduling problem.