The Scent of Bad Psychology

The key takeaways for me from the 80000 hours test (https://80000hours.org/psychology-replication-quiz/):

> Papers that report boring or common-sense results seem to be true very often. Cognitive psychology (e.g. memory and perception) mostly held up.

> ...it turns out for most people "if the claim of the paper doesn't make intuitive sense, the paper is wrong" is a good shortcut that works about 70% of the time.

Generally, the closer to the "metal" (e.g., memory, perception, learning) and the more pedestrian the psychological finding is, the likelier it is to hold good.

I might be giving in to a bit of hindsight bias here, but I have come across this in the psychology of learning myself. It has been a bit underwhelming I must admit. "Good sleep, retrieval practice, elaboration, paying attention, meaningfulness and organization, and spaced repetition result in better learning; massed-practice is not very useful for long-term retention". Well, no kidding!

> a long list of opinions of the importance of credentials

Those haven't been addressed. I think calling them "opinions" is a big stretch; it's a reliable statement of fact that humanity very often relies on the credentials of the speaker, including their training and experience, for evaluating information they communicate.

> What I don't see is any opinion on why the particular arguments are not enough.

It's up to the person making the claims to establish them. We can't spend our time refuting everything anyone says. In fact, making claims in order to force others to refute them is a well-known propaganda technique for creating dysfunction, in part because refuting takes far more time than just making claims. (This article is not propaganda, in case anyone thinks I'm implying that.)

The arguments made by the author are expert judgments; there's little to address by other means.

> it seems that you suggest that we should only discuss articles written by people with credentials on a topic

Personally, I want to spend my time on the best knowledge I can find. Life is short. Identifying quality information depends frequently on credentials; I think that's uncontroversial.

You keep taking things to an extreme - "only" - and to be clear, those are your words and not mine. Somewhere else I also made that explicit. EDIT: Here it is, copied from another subthread to keep everything in one place:

Of course, let's not make this binary, one or the other [merits or credentials]. Everyone uses both to varying degrees, and other signals too (if the post used bad English, for example, many would doubt the author's credibility).

In this case, I lean more heavily on expertise because the content is a product of expertise - rules of thumb. If they wrote 'I can create cold fusion in a teacup', I'd lean heavily on the merits - can they do it or not?

> if it had quiz showing me their method could get over 95% correct after reading a 10 or so word summary of the end result, I'd take it very seriously.

Often, amateurs find seemingly obvious answers to the wrong question. I once was reading Amazon user reviews of translations of Confucius' Analects. For one translation, a few reviewers observed that the translator went bizarrely off-topic in the footnotes, giving his opinions on various things Confucian and contemporary. I ruled out that translation. Then I read a Chinese scholar's review of translations, and learned that such commentary was in fact the standard of the leading editions of the Analects going back millennia. The Amazon users - as amateurs often do - had no idea what they were seeing, and if I hadn't found the scholar's review, I would have believed them too (and missed out what that incredible work translation - a widely recognized work of art and genius - had to offer me from Confucius). That's the last time I trusted Amazon reviews. Similarly, you can find plenty of well-reasoned, persuasive, and totally wrong legal and medical advice online (including on HN).

How can one avoid falling into the same trap as I did with the Amazon reviews? Don't trust amateurs, no matter how persuasive they seem, in questions that require expertise.