Cargo cult science (1974) • Hacker News Board

Re: Cargo cult science (1974)

#161

Post by caddemon » Thu, Mar 09, 2023, 2:21 AM UTC

staunton wrote: post #157

Loquebantur wrote: post #156

Earlier quoted context omitted.

You are right about the incentive structure being a main problem in science. But at the heart of the reasons why this structure is the way it is, you find people's utter lack of knowledge about how science does and should work in the first place. Remarably, nearly all the commenters here betray this very lack of knowledge by talking about it as "philosophy". As if it was some practically irrelevant jibberjabber. Scie…

> at the heart of the reasons why this structure is the way it is, you find people's utter lack of knowledge about how science does and should work I don't think people's lack of understanding scientific methods is the main factor shaping the incentive structure, though it's very interesting how big a role it plays and I would love to hear more detailed arguments. In order to change the system, there must be enough a…

It's harder to game metrics when there is a wider diversity of them and the way in which they are weighted in different decision making committees is different. Because it's hard to simultaneously game a bunch of different metrics, and the benefits from gaming a single one are probably worse than the benefits of just organically doing what you think is best.

But yeah I totally agree that these are extremely hard problems with no silver bullet. Philosophically and from a "how do systems of humans work together" perspective. Teaching methodology of science like a cookbook I also agree is the wrong play and just digging in further to what got us where we are.

I'm curious what makes you say scientists are collaborating more? Stats or has that been your experience? I've seen a lot of semi political collaborations where people are mostly just posturing to get on more papers. I have also seen a decent number of good faith interdisciplinary collaborations, though it seems to be a bit of a coin flip whether they flame out over disagreements. I've personally not seen much cooperation within the same field though that I'd consider genuine.

Thanks for sparking the interesting discussion in any event, it's not often I end up debating people on both sides of a debate so this has been fun.

Re: Cargo cult science (1974)

#162

Post by staunton » Thu, Mar 09, 2023, 11:47 AM UTC

caddemon wrote: post #161

staunton wrote: post #157

Earlier quoted context omitted.

> at the heart of the reasons why this structure is the way it is, you find people's utter lack of knowledge about how science does and should work I don't think people's lack of understanding scientific methods is the main factor shaping the incentive structure, though it's very interesting how big a role it plays and I would love to hear more detailed arguments. In order to change the system, there must be enough a…

It's harder to game metrics when there is a wider diversity of them and the way in which they are weighted in different decision making committees is different. Because it's hard to simultaneously game a bunch of different metrics, and the benefits from gaming a single one are probably worse than the benefits of just organically doing what you think is best. But yeah I totally agree that these are extremely hard prob…

In my field there's lots of honest and fruitful collaboration. It's not all perfect but definitely a good situation. Technology has been making collaboration easier and faster as well. Globalization has led to international scientific collaboration being ever more expected and normal.

Look into the organization of CERN for example. It's a huge project, highly politicized in many ways, with many parties that have their own goals and motives. Yet they all realize that a huge and sufficiently effective collaboration is the only way to do this science at all. And they are doing it.

Re: Cargo cult science (1974)

#163

Post by staunton » Thu, Mar 09, 2023, 12:02 PM UTC

caddemon wrote: post #159

staunton wrote: post #152

Earlier quoted context omitted.

> If you think what I'm proposing is three lectures on philosophy of science than either you have zero idea how a PhD program works or you're arguing in bad faith. Talking about three lectures was meant as a question of what you actually propose (you might say that's bad faith). However, I believe you will find many people who you can convince that this is an important problem and they will suggest the "lecture appro…

Most decisions made about the structure of PhD training are not made by random administrators like undergrad. The head of a program will be a faculty member in that department actively doing research and advising students, usually a faculty member that has been very successful in their research output (at least by official metrics). The decisions on how to structure first year curriculum, what to require for graduati…

> there are also people that are just remarkably unaware of how to be remotely statistically rigorous

That reminds me of D. Kahneman's self-righteous book where he recounts teaching statistics for years "without it ever occuring to him that it might be used in his research". Claiming to have learned this lesson, he proceeds to promote his book while half of it's claims have failed to reproduce. He's still a celebrity im his field.

My point being: you really think this situation ever used to be better?

Re: Cargo cult science (1974)

#164

Post by smogcutter » Fri, Mar 10, 2023, 4:07 AM UTC

low_tech_love wrote: post #127

atleastoptimal wrote: post #86

Earlier quoted context omitted.

Yeah, he seems to be making it a matter of "well the smart people, the ones who really matter, don't need education, so all the sub 99th percentile students don't really matter because the smart ones aren't benefitting" So teaching kids to read, understand logical inference, showing them different parts of the world and how to look at it, doesn't matter because the wonderful gifted kids would have happened upon those…

Thanks for both points, I’ll think about them. However, you basically just confirmed what I argued: it’s impossible to criticize educational methods. Just because education ”can” be useful, that doesn’t mean it ”is” useful. I wish every person in the world could have access to transformative education, but the truth of the matter is that almost no teacher out there can actually do that; most are just very well-intend…

“Your profession is valueless, and you kinky think otherwise because you have an irrational emotional connection to it” is not the incisive, conversation-provoking criticism you seem the think it is.