Young machine learning practitioners : That is why you should do not trust accuracy when doing a ML model on imbalanced datasets.
And in general, better use precision, recall, f1-score or confusion matrices
Heuristics that almost always work
451–460 of 541 posts
Re: Heuristics that almost always work
#452Earlier quoted context omitted.
> the mere act of doing it more often dramatically changes the risk. Kind of. However, you already know that the first N outings didn't have a disaster. So those should be discarded from your analysis. Doing it N times more has a lot of risk, doing it the N+1th time has barely any.
If you'll die if a roll of three dice comes up sixes, you're not really in a lot of danger. If you do it every day, you have about 15 months to live.
Yup. Don't do stuff (repeatedly) that have an absorbing barrier - https://medium.com/ml-everything/nassim-taleb-absorbent-barr...
Re: Heuristics that almost always work
#453Earlier quoted context omitted.
Don't get on that greased sliding board that ends at the top of a cliff. Once you start sliding, it will be hard to stop because of the grease, and then once you slide off then end you will fall and die. Do you really think this slippery slope argument is a fallacy? FWIW, wikipedia acknowledges slippery slope can be a legit argument when the slope, and it's chain of consequences, are actually real. https://en.m.wikip…
From your linked article: > The fallacious sense of "slippery slope" is often used synonymously with continuum fallacy, in that it ignores the possibility of middle ground and assumes a discrete transition from category A to category B. In this sense, it constitutes an informal fallacy. "If you take N steps, you will take N+1 steps" is a fallacy whenever it's possible that you won't take N+1 steps.
Not what was said. What was said: if you DON'T take the Nth step, you then WON'T take the N+1 step.
"Not A -> Not B" is different logic than "A -> B". A is necessary but not sufficient for B.
Re: Heuristics that almost always work
#454I read them as sloppy caricatures that provide little value to the conversation. Let's take the security guard: "The only problem is: he now provides literally no value. He’s excluded by fiat the possibility of ever being useful in any way. He could be losslessly replaced by a rock with the words “THERE ARE NO ROBBERS” on it." That is blatantly not true, the guard provide value since the wanna-be robbers don't know w…
At least in the case of the doctor, the numbers are also way off. A typical doctor sees thousands, if not tens of thousands of patients in their career. This would certainly translate to _way_ more than two or three misdiagnosed cancers.
Definitely. I suppose it's like an on-call engineer with a dodgy metric that "cries wolf" very regularly. There are hundreds of deadly conditions that have "flu-like symptoms", cancers that manifest in a myriad different ways (from anemia to blurred vision), and other serious conditions that start out as plain old lethargy or listlessness and get diagnosed as depression. I have encountered a complacent doctor who misdiagnosed my sister's cancer as "pain" (??) prescribing pain killers for weeks while the cancer spread and additional symptoms appeared. [Edit: I misremembered as months not weeks, but it was about 6 weeks]. (She died). My father is 95 years old, and had extremely low blood pressure for months, making him constantly sleepy. My mother accidentally made the discovery after she bought a blood pressure monitor for my sister (who has high blood pressure). They took him off the medication, and he's back outside chopping small pieces of firewood and taking short walks. As far as the doctor was concerned he was tired because he was old, and his body was "winding down like an old clock". Too lazy to take his fucking blood pressure. Doctors get extremely complacent, burnt-out, depressed etc. They frequently miss the 1:1000 time that the symptoms are something serious, so I literally never trust a doctor to make a diagnosis without carrying out actual tests. If I'm ever ill enough to need one, I'll have done my own research before visiting one. I don't care if it pisses them off.
Re: Heuristics that almost always work
#455Earlier quoted context omitted.
"On your doctor take, you do know that the other author of this post is a licensed and practicing Physician, right?" -- Your observation is indeed much more interesting than the whole article, since it shows a reasonable (and not a caricature with zero value) heuristic. You are saying that I (who may or may not be a physician, but for argument's sake let's say I am not) should not have an opinion that is different, w…
I'm confused. You say it's a bad caricature, then provide an anecdotal example that seems to closely match the caricature.
I think OPs point is that the article ignores the reason why the heuristic is used in the first place, which is to optimize the process at scale.
The real cost of gathering additional data and knowledge about the circumstances in the 0.01% chance you know that you have at this point, at scale, means you'll end up with worse overall outcome by not being able to scale to all events that need the attention.
Now, if a doctor is just being lazy, doesn't check anything, says you'll be fine take Tylenol and then spends the next half hour reading a book until the next appointment. Or simply wants to go through twice as many patients to make more money. Ya sure, that's just being lazy and useless, and negligent, replace them by a rock at that point.
But if there are 100 people with initial symptoms, and only 1 doctor. And the deep dive to properly asses the likelihood of a 0.01% chance event in the case of symptom: "My hip hurts when I walk" takes multiple hours, a lab test, many follow ups, etc. While this happens and maybe out of the 100 patients waiting, some have symptoms like: "I'm actively bleeding out my mouth.". "I have spores on my skin." "I'm in so much pain I can't fall asleep." All with known much higher likelihood of something pretty bad and urgent.
That's why triage is so important. And this use of the heuristic at scale might make sense when considering the cost/time and available resources trade off.
At the individual level, it means eventually someone will get shafted by this, they'll be sent home with Tylenol, and 3 days later will have a stroke and it would have turned out they are the really rare case where hip pain could indicate a risk of stroke due to say a blood cloth.
But at a larger scale, many more people will have received the treatment they needed more urgently.
Re: Heuristics that almost always work
#456Some of these examples are wrong.
Both Skeptic and Futurist provide value for risk-averse people, as they help to explain away the changes. That’s why lots of people read these skeptics and futurists, because their message is predictable and brings very expected emotions every time.
Example with Interviewer assumes finding diamonds among low-promise CVs has value for a company — does not work that way for companies with huge hiring funnels. Interviewer that focuses on effective but imperfect screening will get far in these, all the while bringing lots of value to the company.
Article also simplifies these roles down to a single, almost binary, function. Which is pretty ironic, given the idea this article tries to communicate.
Re: Heuristics that almost always work
#457I read them as sloppy caricatures that provide little value to the conversation. Let's take the security guard: "The only problem is: he now provides literally no value. He’s excluded by fiat the possibility of ever being useful in any way. He could be losslessly replaced by a rock with the words “THERE ARE NO ROBBERS” on it." That is blatantly not true, the guard provide value since the wanna-be robbers don't know w…
It wouldn't change much if the rock said "IT'S NOTHING, TAKE TWO ASPIRIN AND WAIT FOR IT TO GO AWAY AND CALL THIS NUMBER IF IT DOES NOT"
So where can I found the trial for rock-based medical care?
Re: Heuristics that almost always work
#458This story reminded me of a story written by a Czech biologist who studied animals in Papua-New Guinea and went to a hunt with a group of local tribesmen. The dusk was approaching, they were still in the forest and he proposed that they could sleep under a tree. The hunters were adamant in their refusal: no, this is dangerous, a tree might fall on you in your sleep and kill you . He relented, but silently considered…
> for a lifelong hunter, 1:5000 are pretty bad odds There’s a term for this which I’m unable to recall and it’s not easy to Google. Would greatly appreciate if someone could help me out here!!
Is it Micromort? I just learned about this from a comment above.
Re: Heuristics that almost always work
#459Earlier quoted context omitted.
I think you're reading it wrong. After a long life of rock climbing, there's no significant risk of doing it one last time or 10 last times (ignoring the effect of old age itself and whatever). But when you're in earlier stages of your life, you're asking a different question: You're asking, is this something I want to do hundreds or thousands of times in my life, knowing that each of those times has a small chance o…
That would be a reason to have not climbed more than a specific rate ever. It wouldn't be a reason to scale down the rate of climbing as you age.
You're making it sound like it's a decision they made when they got into rock climbing initially, that they would climb frequently while young and then scale back as they get older.
Now, making that decision at the outset does make sense, because it will drastically reduce the number of climbs you make in your life compared to climbing frequently throughout your life, and rock climbing while young is less risky than rock climbing while old.
But importantly, I don't think that's what GP did. It sounds to me like GP spent their youth climbing a lot without considering their mortality, but then decided to scale back because they realized climbing that often for the rest of their life would be dangerous. Maybe they spent the time from 20 to 35 climbing 30 times per year, in keeping with my earlier example. That means they've already climbed 450 times. Risky, but they made it through alive. At 35, they start to consider their own mortality, and they have the choice between climbing 900 more times by keeping to their current rate, and climbing 30 more times by reducing their rate (or something in between). Deciding to scale back makes sense.
There is no logical fallacy.
Re: Heuristics that almost always work
#460This story reminded me of a story written by a Czech biologist who studied animals in Papua-New Guinea and went to a hunt with a group of local tribesmen. The dusk was approaching, they were still in the forest and he proposed that they could sleep under a tree. The hunters were adamant in their refusal: no, this is dangerous, a tree might fall on you in your sleep and kill you . He relented, but silently considered…
It seems like 1:5000 is not an accurate probability, but just a number which he chose in order to convince himself that sleeping under a tree was not a risky activity. But he chose a bad number and later realized that an argument based on that number was not convincing.
If the chance was actually 1:5000, the longevity of trees would be similar to that of hunters sleeping under them - or lower, since hunters can sometimes avoid the hazard, but trees are exposed to it every night (and all day as well). Actual data on tree mortality seems to indicate that the chances are much lower than this. Almost certainly they do not make sleeping under trees a significant risky activity.