Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Science without Validation in a World without Meaning

americanaffairsjournal.org

Post reply on HN
Print view View original

71–80 of 127 posts

Re: Science without Validation in a World without Meaning

#71

Post by cgiles »

memexy wrote: post #56
cgiles wrote: post #44

I work in molecular biology research, and I think this is a great article that strikes at the heart of many problems in the field. I can't comment on the climate change stuff, although I wish he hadn't included it because it was almost certain to distract people from the overall point. The problem is that there are no remotely comprehensive, predictive, and mathematical models of what goes on inside of cells. It is p…

What's missing from current mathematics to make predictive models for biology? I did a search for "neural network cell simulation" and got a few hits, e.g. https://ieeexplore.ieee.org/document/8805421 . So it seems that people are working on the problem of predictability (or at least augmenting the researcher's/experimenter's ability to do some analysis ahead of time based on simplified models).

> What's missing from current mathematics to make predictive models for biology?

Well, I think that, no joke, there is a Nobel Prize waiting for anyone who knows the answer to that. I think this is the next big paradigm shift needed in biology, not to mention several other fields.

Who is to say that the problem is strictly mathematical, though? It could be that the math exists, but no one knows how to fit existing data into it, or it could be that there is not enough data, or the right kind of data, to make such a model yet. It could be that both the data and algorithm exists, but we need to turn the Earth into computronium to run it. Who knows?

> So it seems that people are working on the problem of predictability

I'm sure they are. They have been for decades. The last time I did a systematic review of this area was before the resurgence of neural networks, so I can't really say what is the latest progress, or whether the progress in ANNs can inform this problem. I suspect it's very possible.

The situation right now, as far as I know is that: A) most biologists don't even know this is a problem, and B) those who do, don't have any idea what the solution is, or if one even exists (note the author of the linked article was pessimistic on that point).

Re: Science without Validation in a World without Meaning

#72

Post by mturmon »

cgiles wrote: post #44

I work in molecular biology research, and I think this is a great article that strikes at the heart of many problems in the field. I can't comment on the climate change stuff, although I wish he hadn't included it because it was almost certain to distract people from the overall point. The problem is that there are no remotely comprehensive, predictive, and mathematical models of what goes on inside of cells. It is p…

Thanks for this generous comment. The author of TFA has articulated a genuine problem that is central to many large-scale investigations these days, across many domains. We rely a lot on complex computer simulations, or complex physics-based models, that have a lot of fiddly details that are understood by only a limited set of people.

Yet, we want to learn from these models, and we want to reach conclusions from them. This has turned into a key problem for the scientific enterprise.

There are so many linked issues, some technical, some philosophical: Mere Monte Carlo state exploration is wasteful and doesn't provide much insight. Often we don't have error bars on model outputs to even know if an "improvement" in a metric is significant. There can be unknown unknowns that keep us from trusting our models completely.

It's a very rich and challenging problem space.

In my understanding, the Dept. of Energy was the first community to engage with these problems due to the test ban treaty. They had the mandate to ensure the nuclear stockpile works, despite not being able to fully test it. So they need models and they need to know how far to trust them.

One landmark reference for that is the NAS report on uncertainty quantification and complex models: https://www.nap.edu/catalog/13395/assessing-the-reliability-...

Re: Science without Validation in a World without Meaning

#73

Post by themodelplumber »

cgiles wrote: post #44

I work in molecular biology research, and I think this is a great article that strikes at the heart of many problems in the field. I can't comment on the climate change stuff, although I wish he hadn't included it because it was almost certain to distract people from the overall point. The problem is that there are no remotely comprehensive, predictive, and mathematical models of what goes on inside of cells. It is p…

Thanks for sharing your insights and experience. Are there predictive models elsewhere in your field, or in science, that inspire this search? Also I'm wondering if there are predictive models for cellular sub-systems, for example--the simpler stuff (afaik).

Re: Science without Validation in a World without Meaning

#74

Post by s1artibartfast »

snowwrestler wrote: post #30
btrettel wrote: post #26

Earlier quoted context omitted.

Well, the following seems fairly harsh to me, and I would recommend rephrasing it: > IMO few categories of professions have a harder time understanding cutting-edge science than engineers. That is because they think they know science because they use similar mathematical and technical tools, when in fact the professions do the exact opposite of one another. The wording "few categories" in particular seems hyperbolic.…

I think the huge overlap in shared tools, techniques, technologies, language etc. can act to obscure the fundamental differences in mindset between the fields. If a research scientist doesn't know how their work is going to turn out, they're probably on the right track. If an engineer doesn't know how their work is going to turn out, they're probably on the wrong track. Non-STEM folks might have a hard time understan…

bringing this back, do you think the mindset of policy makers be closer to that of the scientist or engineer?

Re: Science without Validation in a World without Meaning

#75

Post by Sophistifunk »

cgiles wrote: post #44

I work in molecular biology research, and I think this is a great article that strikes at the heart of many problems in the field. I can't comment on the climate change stuff, although I wish he hadn't included it because it was almost certain to distract people from the overall point. The problem is that there are no remotely comprehensive, predictive, and mathematical models of what goes on inside of cells. It is p…

Is it nobody knows how to make the jump from basic physics to predicting cell behaviour (sorta like quantum/classical), or would predicting the behaviour just require too much compute power?

Re: Science without Validation in a World without Meaning

#76

Post by bsder »

pjkundert wrote: post #45
bsder wrote: post #43

Earlier quoted context omitted.

> thus lowering the CFR (Case Fatality Rate) for that segment of the population to something near a low multiple of the normal Flu -- the "lockdown" hypothesis should have been discarded, violently, for that segment of the population! You are deeply into "Not even wrong" territory. The plural of anecdote is not data. Did you forget HCQ already? Lots of "positive reports" and yet controlled studies shows that it kills…

OK, I'll bite, and ignore the dismissive "not even wrong" insult! Remember how the scientific method works: if you make a hypothesis, it's the most fragile thing in the universe -- a single falsification is all it takes to send you packing. So, every time someone trots out "the plural of anecdote is not data" when presented with a refutation, this tells us something important... Like it or not, that's just how it wor…

> A group's CFR is almost the only thing that matters to that specific group.

Except that you don't know what defines a group.

We know what makes someone at risk. That does NOT imply the reverse--that we know what makes someone NOT at risk.

Sure, maybe 18-30 year olds by and large don't drop dead. That doesn't mean they don't wind up with stroke risk or a damaged lifespan. And, maybe some 2-3% subgroup of them drops dead because they have a particular receptor--oops, sucks to be you but we can't undo it now that we let the pandemic loose.

You don't just shove a disease through a population when you don't know what the effects are. That is unethical and immoral at the level of eugenics. Good God, man.

We actually did this sort of thing with chicken pox and now those children get the joy of shingles possibly blinding them in their adult life. Oops.

And, by the way, you know what disease we achieved herd immunity to before vaccines? Oh, yeah, NONE. This one will be no different.

Re: Science without Validation in a World without Meaning

#77

Post by guscost »

dang wrote: post #70
guscost wrote: post #6

Earlier quoted context omitted.

If science is truly what you describe, then it has no place in the making of public policy. Never again.

You may have a good point, but this is a bad HN comment. For it to be a good HN comment you'd have to develop the point (1) in more detail, (2) with more explanation, and (3) without denunciatory rhetoric. If you were to do that, I'd be interested in reading it. To post like this is against your interest because, to the extent that readers can tell what you're saying, they'll just be turned against it by the grandios…

Thank you for the feedback (and for all the moderation work). It can be hard to detach and focus on the “purely informational” substance of an argument, but you are right that that is pretty much always for the best. This is going to sound like such a HN cliche, but if there was a good way to convey emotion as independent metadata instead, I’d try to use that.

And to clarify my comment, I’m frustrated to see so much of our public discourse still rely on appeals to authority, even when we’re talking about claims that are fundamentally impossible to validate. Authority figures are proving to be nearly-as-bad at navigating these difficult topics as the rest of us, but it seems that to do any better, the public would need to understand the actual ideas being discussed, which seems a long way off (I don’t even hear this mentioned as a possible goal).

So for now we are left with a choice between trusting what is essentially an order of clerics, empowering another authority that won’t do much better (regular old politics), or mob rule. I have to go with the second option, but it is not much better than the others, at best.

Re: Science without Validation in a World without Meaning

#78

Post by AlexCoventry »

cgiles wrote: post #44

I work in molecular biology research, and I think this is a great article that strikes at the heart of many problems in the field. I can't comment on the climate change stuff, although I wish he hadn't included it because it was almost certain to distract people from the overall point. The problem is that there are no remotely comprehensive, predictive, and mathematical models of what goes on inside of cells. It is p…

Hasn't it always been like that? It took like fifteen years of sustained investigation just to identify DNA as the substrate of genetic transmission in bacteria. We've come a long way since then, but isn't it a bit much to expect that we'd already have effective, automated, catholic simulations of cellular biology?

Re: Science without Validation in a World without Meaning

#79

Post by guscost »

dang wrote: post #67
guscost wrote: post #37

Earlier quoted context omitted.

We can't easily validate models of reality that are complex/stochastic, because any experimental result could be a statistical anomaly. In particular this is a problem for models of long-term and/or global phenomena like climate or public health, since we only get one "experimental run", so to speak. Therefore: > Confronting the problems of complexity, validation, and model uncertainty, I have previously identified f…

I haven't read the article, but #4 is the interesting item on that list. #3 is the Goldilocks option, the one that sounds between-the-extremes like the just-right porridge that Goldilocks ate—but it's not realistic, because "be extremely cautious when interpreting them" exceeds what the human mind is capable of.

It may be possible for a person to succeed with #3, but I have a hard time believing it could work in a community of people who disagree.

For #4, the closest thing I can imagine is a utilitarian approach, which of course I’m going to prefer as an engineer, and which (I think) ultimately reduces to #1. That is a tough problem I don’t expect to see solved in my lifetime, but I’d be happy to be proven wrong.

Re: Science without Validation in a World without Meaning

#80

Post by renewiltord »

pdonis wrote: post #5
snowwrestler wrote: post #4

An article by an electrical engineer, published in a political journal, about climate change... I had low hopes, which were met. IMO few categories of professions have a harder time understanding cutting-edge science than engineers. That is because they think they know science because they use similar mathematical and technical tools, when in fact the professions do the exact opposite of one another. Engineers use wh…

> Working without knowing whether you're correct is not epistemological conflict. It's the fundamental condition of being a scientist. This is true, but it's also true that, when making public policy, we should be thinking like engineers, not scientists. Public policy is not about constructing new knowledge. It's about working out conflicts between different values and priorities based on our best current knowledge.…

It's just a status quo bias to choose "act to preserve without prior of belief > 0.95". It's a novelty bias to choose "act to change without prior of belief > 0.95". But there are many modes of action in between. Knowledge is necessarily imperfect (cf. Conjectures and Refutations). The engineer/scientist difference isn't all that useful in reality. Even disciplines dominated by engineering rapidly see advances over time purely through process improvements which often manifest across the field as the hypothesis-test-apply loop of knowledge acquisition.

Public policy, like all decision making, should work with a gestalt view of utility and certainly most organizations attempt to do that irrespective of whether their spokespeople claim otherwise. It's like any intervention: if the patient will almost certainly die if you do nothing, you try the most unlikely things (assuming consent) but if the patient will most certainly recover with no ill effects you are not going to try an experimental intervention. And naturally you do account for change fatigue, etc. etc. Put simply, the science of policy is not as puerile as you think it is, but the nature of its execution is necessarily a complex interweave of the power centres available and their objectives and influence on the policymaking body.

After all, we don't all have the same objectives.

Post reply on HN