Live data from Hacker News

An erroneous paper on religion and generosity is retracted

psychologytoday.com

101–110 of 165 posts

Re: An erroneous paper on religion and generosity is retracted

#101
post #42

I think it's situations like this that result in society having such a hard time with 'science'. The term has been so heavily co-opted by fields that just don't have sufficient rigger for the term to hold weight. Yet, on various topics, we have this publicized attack of "your a science denier!". At the end of the day, there are two 'types' of science, one where I can take the results and make accurate predictions, an…

Have you never tried to use a tool from a cs paper and found that their code didn't even build or barely worked? This isn't a question of rigorous and non-rigorous fields. This is a challenge with artifact evaluation in all fields.

I'd encourage you to read some of these 'papers', there is often nothing to evaluate, nothing to make predictions on. Usually, just a survey and a statistical correlation test.

Re: An erroneous paper on religion and generosity is retracted

#102
post #64
post #42

I think it's situations like this that result in society having such a hard time with 'science'. The term has been so heavily co-opted by fields that just don't have sufficient rigger for the term to hold weight. Yet, on various topics, we have this publicized attack of "your a science denier!". At the end of the day, there are two 'types' of science, one where I can take the results and make accurate predictions, an…

Scientists usually try to distance themselves by saying those are soft science or even pseudo-science. This leads to the embarrassment of the demarcation problem[1] which is that no one can give a bright-line rule[2] to distinguish between the "real" science and pseudo-science. All of the demarcation criteria that have been proposed (such as Popper's falsifiability[3]) are inadequate in one way or another. In particu…

> which is that no one can give a bright-line rule[2] to distinguish between the "real" science and pseudo-science

"if you can't repeat and predict, stop calling it science" seems like a nice bright line.

Peer-review obviously isn't enough, I'd like to see peer-replicated studies become a thing.

Re: An erroneous paper on religion and generosity is retracted

#103
post #64
post #42

I think it's situations like this that result in society having such a hard time with 'science'. The term has been so heavily co-opted by fields that just don't have sufficient rigger for the term to hold weight. Yet, on various topics, we have this publicized attack of "your a science denier!". At the end of the day, there are two 'types' of science, one where I can take the results and make accurate predictions, an…

Scientists usually try to distance themselves by saying those are soft science or even pseudo-science. This leads to the embarrassment of the demarcation problem[1] which is that no one can give a bright-line rule[2] to distinguish between the "real" science and pseudo-science. All of the demarcation criteria that have been proposed (such as Popper's falsifiability[3]) are inadequate in one way or another. In particu…

even statisticians call social science - psychology especially - a pseudo science

Re: An erroneous paper on religion and generosity is retracted

#104
post #62
post #42

I think it's situations like this that result in society having such a hard time with 'science'. The term has been so heavily co-opted by fields that just don't have sufficient rigger for the term to hold weight. Yet, on various topics, we have this publicized attack of "your a science denier!". At the end of the day, there are two 'types' of science, one where I can take the results and make accurate predictions, an…

For me personally, situations like this make me trust science that much more. If someone finds that a previously published paper contained errors, as any human work is likely to, and the response is to go back and correct those errors or publicize whatever was wrong with the research, well that's science working as intended. That's freaking awesome. What I get really, really, really frustrated with is the attitude th…

Scientists bear a public responsibility. Studies like these are used to justify public policy, opinion pieces, etc. If a civil engineer makes a mistake that impacts the public (say a malfunctioning building structure) that could have been prevented were it not for lack of controls, he or she would be investigated, and potentially be basically blacklisted in their field. If a doctor made a negligent mistake because he was unaware of how to use a tool and a patient injured, he would be held accountable.

In this case, social science analyses are used to answer questions of major public importance. Governments are constantly trying to reduce suicide rates, and make their populaces happier. Papers making claims that religion makes children selfish and unhappy are used to make public policy. In this case, if this paper were used as justification for legislation, we now know that the policies it would tend to suggest would be bad for the population. Someone, somewhere probably ought to be hold accountable in the same way as any other professional. I do hope that journals take appropriate precautions with this researcher in the future, and that the peer reviewers assigned to this case are duly sanctioned. This is a complete failure as professionals.

There are situations in which scientists can and do get things wrong through no fault of their own. For example, during the highly publicized EM drive tests a while back, an initial NASA report indicated that thrust was observed after careful evaluation. This is fine... they reported what they saw. However, after some additional engineering and measurement tuning and stronger sensors, the thrust was attributed to another source, so the claims were retracted. This is science. At every step the scientists demonstrated competence and professionalism. Nowhere did anyone say 'oops we forgot to use the sensor the right way that's why it didn't work, and in the meantime our paper was used to engineer other solutions'. There is a fundamental difference between being wrong and misrepresenting what you saw, whether through mistake or ignorance.

Re: An erroneous paper on religion and generosity is retracted

#105

My default presumption is that all results from the "social sciences" are false if they contradict my intuition. Whatever these fields are producing, it's not science in the Popperian or practical sense. A ton of policy has been built on the bad research and wishful thinking of the past few decades, and it's going to take a long time to unwind it all. The problem, really, has been accelerating for a while. Something…

My theory is that it's engineering, not science, that confirms our knowledge. You don't really know something about the world until you can use it to do things. WWII-era science was being incorporated into tools (well, weapons), so it had to work.

Re: An erroneous paper on religion and generosity is retracted

#106
post #95

Earlier quoted context omitted.

Your argument seems to be purely emotional disdain. > Yet it seems to happen far less often Citation needed. And if so, that is a reason to draw a line where on one side is "not-science"? That is just absurd. Does a car A stops being a means of transport when it subjectively breaks down more often than car B? > very least, in less publicly impacting ways Excuse me? * A literal century ago someone failed to translate…

You're supporting my point here, all your examples did not make an accurate prediction. Thus, they are not known 'facts'. Your definition of 'science' is 'the best we know', mine, and I think what is meaningful for public discourse is 'this is true'.

Sure, but what's your definition of "true"?

Derivable from first principles? General consensus? Observed once and seems to fit with the current model?

Re: An erroneous paper on religion and generosity is retracted

#107
post #102
post #64

Earlier quoted context omitted.

Scientists usually try to distance themselves by saying those are soft science or even pseudo-science. This leads to the embarrassment of the demarcation problem[1] which is that no one can give a bright-line rule[2] to distinguish between the "real" science and pseudo-science. All of the demarcation criteria that have been proposed (such as Popper's falsifiability[3]) are inadequate in one way or another. In particu…

> which is that no one can give a bright-line rule[2] to distinguish between the "real" science and pseudo-science "if you can't repeat and predict, stop calling it science" seems like a nice bright line. Peer-review obviously isn't enough, I'd like to see peer-replicated studies become a thing.

What about astronomy, cosmology, archeology, paleontology, volcanology, evolutionary biology, cladistics, macroeconomics, etc., which do not allow us to "repeat and predict," as you say?

Some of these cases can be rescued by considering "retrodiction"[1][2] as valid substitute for prediction in the right circumstances, but not all.

I personally think the analysis of the Mott problem[3] points the way to the solution to some of these kinds of issues. That is, a prediction can take the form of a likelihood function which assigns high probabilities to certain combinations of events and low probabilities to others. Theories with low perplexity[4] can be considered correct even if they can't make predictions, and the study of such theories can be scientific. But as far as I know I am the only one who thinks so.

[1]: https://en.wikipedia.org/wiki/Retrodiction

[2]: https://afdave.wordpress.com/2007/09/04/sir-karl-popper-and-...

[3]: https://en.wikipedia.org/wiki/Mott_problem

[4]: https://en.wikipedia.org/wiki/Perplexity

Re: An erroneous paper on religion and generosity is retracted

#108
post #39

Earlier quoted context omitted.

Simple...data representation is not the same as data meaning. I teach an introductory stats course and we hammer this in. Categorical data are often represented as numbers or other short indicators for storage purposes. Typically I fmultiple choice the encoding is by the order of the choice options. I not infrequently see average of gender because male = 0 and female = 1 or vice versa and someone generates a table wi…

The bigger issue here seems to be the use of ordinals in the data collection process. For instance, a lot of my CSVs don't have them and R and pandas are perfectly capable of enumerating. Why do you even need to put ordinals in the dataset? Does excel want this sort of thing or something?

Your don't need to people just do... I've kinda always assumed or connects to olden days and efficient storage and memory use. Male is a four character string, 1 is an integer.

Re: An erroneous paper on religion and generosity is retracted

#109
post #106
post #95

Earlier quoted context omitted.

You're supporting my point here, all your examples did not make an accurate prediction. Thus, they are not known 'facts'. Your definition of 'science' is 'the best we know', mine, and I think what is meaningful for public discourse is 'this is true'.

Sure, but what's your definition of "true"? Derivable from first principles? General consensus? Observed once and seems to fit with the current model?

I don't really now how to make it more clear: Can I make accurate predictions from it? That's it.

Re: An erroneous paper on religion and generosity is retracted

#110
post #92
post #64

Earlier quoted context omitted.

Scientists usually try to distance themselves by saying those are soft science or even pseudo-science. This leads to the embarrassment of the demarcation problem[1] which is that no one can give a bright-line rule[2] to distinguish between the "real" science and pseudo-science. All of the demarcation criteria that have been proposed (such as Popper's falsifiability[3]) are inadequate in one way or another. In particu…

Why not have a general checklist with a minimum set of requirements for scientific papers that are relevant across all branches of science? The people putting their names on the paper would have to show that they followed everything or give reasons for skipping a step. The receiving journals would have their editors re-check the checklist. As part of reporting the results of the paper, the level of completeness of th…

There are things like CONSORT[1] which kind of do this. Statisticians like Fisher[2] have a ton of good general advice of the design of experiments. (A plug for The Lady Tasting Tea[3] and the 7 Pillars of Statistical Wisdom[4] feels appropriate here.)

On the whole though, most of the things you should and should not do are so domain specific its very hard to give much useful advice at the level of "all science." Right now this seems to work because researchers are so eager to anticipate objections and and avoid unnecessary arguments during peer review they stick slavishly stick to the same methods used by seminal papers in their field, and this has the same effect as running down a checklist.

There probably is a case to be made for using an actual checklist, though[5].

[1]: http://www.consort-statement.org/

[2]: https://en.wikipedia.org/wiki/Design_of_experiments#Fisher's...

[3]: https://en.wikipedia.org/wiki/The_Lady_Tasting_Tea

[4]: https://www.goodreads.com/book/show/27311742-the-seven-pilla...

[5]: http://atulgawande.com/book/the-checklist-manifesto/

Post reply on HN