Science without Validation in a World without Meaning • Hacker News Board

Re: Science without Validation in a World without Meaning

#81

Post by guscost » Wed, Jun 03, 2020, 2:35 AM UTC

pjkundert wrote: post #50

guscost wrote: post #23

Earlier quoted context omitted.

The botched response to COVID-19 will bring the whole institution of modern science crashing down, when all is said and done. Maybe you don't want to believe that, but there it is: https://guscost.com/2020/05/12/pandemic-woo/ By the way, I still have not heard of a single serosurvey being run in South Korea. Why is that?

Why, indeed. It's almost like people just don't want to know the CFR of this thing... Why wouldn't you want to know the CFR, in exquisite detail, down to groups refined by as many distinguishing details as you can possibly measure? But, surprise! You simply cannot know the CFR of Covid-19 without statistically valid immunity studies of the general population. Which nobody is doing! Just ... crazy. Its stunningly clea…

> In 2-3 weeks, a brutal "second wave" is just ... guaranteed. And, it'll be presented as a "surprise"! Who could have possibly predicted it! By golly, we'll have to really put the hammer down, this time!

I don’t think this will happen in the northern hemisphere, thankfully. There is reason to believe both that effective R drops off a cliff in the summer months, and that we’re much closer to herd immunity (or whatever is possible with this kind of virus) than most people think. If you want references I can dig them up, but I guess we’ll see either way in a few weeks.

Re: Science without Validation in a World without Meaning

#82

Post by war1025 » Wed, Jun 03, 2020, 3:13 AM UTC

scottishcow wrote: post #28

snowwrestler wrote: post #4

An article by an electrical engineer, published in a political journal, about climate change... I had low hopes, which were met. IMO few categories of professions have a harder time understanding cutting-edge science than engineers. That is because they think they know science because they use similar mathematical and technical tools, when in fact the professions do the exact opposite of one another. Engineers use wh…

You make it sound as if the author is some middling engineer building smartphones for Apple. An EE professor at a well-regarded research university deals with the scientific method all the time . Whatever opinion you may have about the article, as far as I can tell the author is certainly not on "the sidelines" and more than qualified to write about the state of science.

> some middling engineer building smartphones for Apple.

> An EE professor at a well-regarded research university

Oddly enough, I'd trust the experience of an engineer over that of a professor basically every time.

Re: Science without Validation in a World without Meaning

#83

Post by cgiles » Wed, Jun 03, 2020, 3:18 AM UTC

mturmon wrote: post #72

cgiles wrote: post #44

I work in molecular biology research, and I think this is a great article that strikes at the heart of many problems in the field. I can't comment on the climate change stuff, although I wish he hadn't included it because it was almost certain to distract people from the overall point. The problem is that there are no remotely comprehensive, predictive, and mathematical models of what goes on inside of cells. It is p…

Thanks for this generous comment. The author of TFA has articulated a genuine problem that is central to many large-scale investigations these days, across many domains. We rely a lot on complex computer simulations, or complex physics-based models, that have a lot of fiddly details that are understood by only a limited set of people. Yet, we want to learn from these models, and we want to reach conclusions from them…

> Mere Monte Carlo state exploration is wasteful and doesn't provide much insight. Often we don't have error bars on model outputs to even know if an "improvement" in a metric is significant.

The funny thing is, I didn't check the author's name until just now. Ed Dougherty, who people below have derided as a "mere engineer", has been working on these problems forever. I'm honestly surprised he's still active or even alive: he was a graybeard when I heard his talk a decade ago. He is a bona fide systems biologist, one of the oldest ones.

At that time, his group was doing gene regulatory network inference on gene expression with ~600 genes. They were using the kind of approach (MC) you mention to infer a small subset of the overall network.

The main thing I took away from their results (at the time) is you can get multiple drastically different network topologies all with similar metrics on the objective function. This implies GRN inference was not inferring some kind of underlying reality. It also suggests you cannot accurately infer subnetworks, which in turn suggests cellular networks aren't all that modular.

Therefore, really a distinction should be drawn between models that are simply predictive and those that also model the underlying reality, which is even harder.

> We rely a lot on complex computer simulations, or complex physics-based models...we want to learn from these models, and we want to reach conclusions from them.

Not in molecular biology. There genuinely are no models like that except in very limited subfields like protein folding, and 99% of biologists would see them as mathematical mumbo-jumbo.

I see from your bio you're also in engineering research. You would not believe it if I told you how mathematically illiterate the average PhD biologist is. My PhD alma mater added a statistics course for the first time last year, a 2 week summer course. Calculus I is "recommended" for admission. This is not unusual.

It isn't seen as needed, because state of the art research is basically all qualitative, with a quantitative veneer of t-tests overlaid on top. So I'm glad to hear other fields at least recognize the problem. Biology hasn't even got that far.

Re: Science without Validation in a World without Meaning

#84

Post by btrettel » Wed, Jun 03, 2020, 3:32 AM UTC

guscost wrote: post #79

dang wrote: post #67

Earlier quoted context omitted.

I haven't read the article, but #4 is the interesting item on that list. #3 is the Goldilocks option, the one that sounds between-the-extremes like the just-right porridge that Goldilocks ate—but it's not realistic, because "be extremely cautious when interpreting them" exceeds what the human mind is capable of.

It may be possible for a person to succeed with #3, but I have a hard time believing it could work in a community of people who disagree. For #4, the closest thing I can imagine is a utilitarian approach, which of course I’m going to prefer as an engineer, and which (I think) ultimately reduces to #1. That is a tough problem I don’t expect to see solved in my lifetime, but I’d be happy to be proven wrong.

On reflection, option 1 includes like Taleb's philosophy. At first I wrote it off as leading to arguments like "no one knows so let's do what I, an 'expert' say", which I hate. But it's written generically enough to include rules like Minimax, where you don't need model estimates to make a decision. This rule is very conservative, though.

I think there are a fair number of people who succeed at option 3. I can think of a few people who I believe do option 3 well, but as you indicated, they're individuals. I think that the US Department of Energy is probably the closest organization that I'm aware of to succeed at this. (Or at least some DOE labs; I can say from personal experience that groups at both Los Alamos and Sandia take this fairly seriously, though I don't think it has fully permeated their culture.) I'll have to think more about this.

Option 3 is my basic approach, I'd like to think that I succeed at it, e.g.: https://news.ycombinator.com/item?id=23397785

A major problem with option 3 is that it runs the risk of people claiming to do option 3 but actually doing option 2. I think this happens regularly but it's due to ignorance, not malice. Ultimately I think we need to change scientific standards and the STEM curriculum before option 3 becomes tractable on a large scale, but even then I'm not sure it'll work because it'll always be easier to claim to do option 3 while actually doing option 2.

I agree with the writer that option 4 is more of a long-term goal. I wish the comments on this article focused more on #4 than what was discussed...

Re: Science without Validation in a World without Meaning

#85

Post by redis_mlc » Wed, Jun 03, 2020, 4:05 AM UTC

guscost wrote: post #81

pjkundert wrote: post #50

Earlier quoted context omitted.

Why, indeed. It's almost like people just don't want to know the CFR of this thing... Why wouldn't you want to know the CFR, in exquisite detail, down to groups refined by as many distinguishing details as you can possibly measure? But, surprise! You simply cannot know the CFR of Covid-19 without statistically valid immunity studies of the general population. Which nobody is doing! Just ... crazy. Its stunningly clea…

> In 2-3 weeks, a brutal "second wave" is just ... guaranteed. And, it'll be presented as a "surprise"! Who could have possibly predicted it! By golly, we'll have to really put the hammer down, this time! I don’t think this will happen in the northern hemisphere, thankfully. There is reason to believe both that effective R drops off a cliff in the summer months, and that we’re much closer to herd immunity (or whateve…

> we’re much closer to herd immunity (or whatever is possible with this kind of virus) than most people think

Certainly for SF/LA, which is basically an extension of China because of the daily flights to SFO and LAX.

The SF tests show 3.6% of 78,000 exposed to corona, but nobody knows how accurate those tests are. The hospital dashboard has been flat.

I just checked, and the numbers have been flat (no growth) for over a month, with only 62 deaths to date and 38 in ICU now:

https://www.sccgov.org/sites/covid19/Pages/dashboard.aspx#ho...

Re: Science without Validation in a World without Meaning

#86

Post by infogulch » Wed, Jun 03, 2020, 4:35 AM UTC

bobbydroptables wrote: post #59

pdonis wrote: post #52

Earlier quoted context omitted.

> Everything the government does is a subsidy of something or other. No, a "subsidy" is not the same as a purchase. Purchasing something at whatever the current market price is is just a purchase. Fixing prices at lower than the current market price, and making up the difference in various hidden ways, which is what the government does with fossil fuels, is a subsidy. > Building out roads nice new roads in every podu…

>No, a "subsidy" is not the same as a purchase. Did I a say "subsidy" was the same as a purchase? Not sure what you're getting at. >Purchasing something at whatever the current market price is is just a purchase. Um. It's a purchase. It can also be a subsidy and generally is when the government does it. If the government purchases a bunch of oranges to give away (or throw in the gutter) it is subsidizing the orange m…

"Negative externalities" seems to be the operating pivot of this conversation. It seems to me that many laws are a system to identify entities producing negative externalities, and make them bear the cost of that externality. The purpose of the lawmaking process, then, is to be a system that discovers new externalities, or more precisely, defines what is a negative externality and what is not, as it relates to chosen policy.

At what point in the progression of scientific consensus does evidence for the consideration of a new externality require a response by passing laws that define the externality and how it can be bourne?

One argument is that there is not enough scientific basis to ground policy regarding CO2, specifically. And, separately, there is enough evidence that some forms carbon energy should be restricted, but due to other factors like pollution from coal. This argument is coming from a strong negative-rights model of government (like the US), where axiomatically people are allowed to do anything not currently restricted by law. The advantage of this system is that it allows people to act in the face of ever-changing circumstances of the world without needing to get approval from the government every time a new thing is discovered. The disadvantage is that now you have to hold externalities to higher bar of proof.

I think that we, individually and as a society, still have to act in the world, imperfect information and all. We cannot demand perfect data to base our decisions on, because always waiting for perfect data means that every decision will be too late. But, just like in science, we need to be able to change our beliefs/laws as evidence mounts that the basis for our previous belief is wrong. The problem is that politicial discourse is so deadlocked on pure narratives (all negative or all positive, no room for nuance or complexity) that we'll never be able to agree.

Re: Science without Validation in a World without Meaning

#87

Post by logicchains » Wed, Jun 03, 2020, 4:38 AM UTC

bsder wrote: post #76

pjkundert wrote: post #45

Earlier quoted context omitted.

OK, I'll bite, and ignore the dismissive "not even wrong" insult! Remember how the scientific method works: if you make a hypothesis, it's the most fragile thing in the universe -- a single falsification is all it takes to send you packing. So, every time someone trots out "the plural of anecdote is not data" when presented with a refutation, this tells us something important... Like it or not, that's just how it wor…

> A group's CFR is almost the only thing that matters to that specific group. Except that you don't know what defines a group . We know what makes someone at risk. That does NOT imply the reverse--that we know what makes someone NOT at risk. Sure, maybe 18-30 year olds by and large don't drop dead. That doesn't mean they don't wind up with stroke risk or a damaged lifespan. And, maybe some 2-3% subgroup of them drops…

>You don't just shove a disease through a population when you don't know what the effects are.

Nobody is "shoving a disease"; the disease is naturally progressing, that's what diseases do. Most moral systems distinguish between actively causing harm and not doing something that could prevent harm.

Not only that, but while we may not know exactly what the effects of the disease are, we know exactly what the effects of the lockdown are, and it's causing immense misery to many many people. So it's a case of weighing up something of unknown badness (the disease passing through the population) against something with known, definite badness (the lockdowns). So we need to be reasonably sure that the unknown badness of the disease is sufficiently bad to outweigh the known badness of the lockdown.

Re: Science without Validation in a World without Meaning

#88

Post by bobbydroptables » Wed, Jun 03, 2020, 5:38 AM UTC

infogulch wrote: post #86

bobbydroptables wrote: post #59

Earlier quoted context omitted.

>No, a "subsidy" is not the same as a purchase. Did I a say "subsidy" was the same as a purchase? Not sure what you're getting at. >Purchasing something at whatever the current market price is is just a purchase. Um. It's a purchase. It can also be a subsidy and generally is when the government does it. If the government purchases a bunch of oranges to give away (or throw in the gutter) it is subsidizing the orange m…

"Negative externalities" seems to be the operating pivot of this conversation. It seems to me that many laws are a system to identify entities producing negative externalities, and make them bear the cost of that externality. The purpose of the lawmaking process, then, is to be a system that discovers new externalities, or more precisely, defines what is a negative externality and what is not, as it relates to chosen…

Yeah you're pretty much right. But this can be solved pretty simply with a pigovian tax.

It's not inconsistent with any definition of rights that I'm aware of to say that you can't pollute our air without consequences. By contrast, the current state of affairs does not mesh with any philosophical system that I've found. Under what theory can some stranger pollute my air? That's no more justifiable than me pouring perchlorate in my neighbor's well. The stronger someone believes in individual rights, the stronger they support my argument.

I think it's a misreading of philosophy (not saying you're doing this) to say that we need to justify restrictions on obvious negative externalities like air pollution. The polluter needs to justify his actions, not the neighbor whose air is being poisoned.

(The science on the deleterious effects of air pollution is of course settled regardless of what anyone thinks about global warming.)

Re: Science without Validation in a World without Meaning

#89

Post by viburnum » Wed, Jun 03, 2020, 5:59 AM UTC

snowwrestler wrote: post #4

An article by an electrical engineer, published in a political journal, about climate change... I had low hopes, which were met. IMO few categories of professions have a harder time understanding cutting-edge science than engineers. That is because they think they know science because they use similar mathematical and technical tools, when in fact the professions do the exact opposite of one another. Engineers use wh…

So many prominent creationists are engineers. It’s weird.

Re: Science without Validation in a World without Meaning

#90

Post by dnautics » Wed, Jun 03, 2020, 6:06 AM UTC

Sophistifunk wrote: post #75

cgiles wrote: post #44

I work in molecular biology research, and I think this is a great article that strikes at the heart of many problems in the field. I can't comment on the climate change stuff, although I wish he hadn't included it because it was almost certain to distract people from the overall point. The problem is that there are no remotely comprehensive, predictive, and mathematical models of what goes on inside of cells. It is p…

Is it nobody knows how to make the jump from basic physics to predicting cell behaviour (sorta like quantum/classical), or would predicting the behaviour just require too much compute power?

That's what everyone tries to do but it's fatally flawed, especially since the practitioners are often clueless about some of the soft emergent behaviours in between (like chemistry). As Sydney Brenner quipped, modern systems biology is low input, high throughput, no output science., Or as a cs person might say, GIGO principle in action.