> the literature often fails to be as scientifically rigorous
I don’t think that’s true. I think the sciences of psychiatry/psychology/neurology have several problems that are very unique that make them challenging fields. The literature is scientifically sound in methodology and data-analysis, but these challenges have the field progress slower, and have many false paths.
1. The brain is inherently complex. It might not be the most complex thing we study scientifically, but it’s complexity is a very big issue.
2. It’s really easy for the third-variable problem to have in impact in psychology/psychiatry. A classic example of this is the generalization problem. A study done in the US or France or another Western Democratic Educated country often will have findings that do not apply in Asia or African countries. Large swaths of the literature have questionable generalization, to the point this is often acknowledged inside the literature itself.
3. We have limited tools for studying these fields. The gold standard, the FMRI, requires people to be completely still and in a very artificial environment. This clearly will influence test results. This lack of tools is everywhere. We cannot directly measure happiness. We can ask people if they are happy. We can ask their friends and family if they seem happy. We can measure how much time they smile. If our sample size is large enough we can compare life outcomes like suicide. Every way we have to measure happiness has flaws, which makes drawing conclusions harder.
4. Ethics plays a huge role. We cannot, for example, randomly assign 100 children to play violent video games for 20 hours a week from age 10-18 and assign 100 children to never play these games. Thus any data we have about violent video games and children will not be as clean. We can assume that children that decide to play violent video games probably share other traits and would differ in a lot of ways to children that do not decide to play violent video games. Thus, it will be hard to determine if the violent video games had any impact, or if it was a third variable that many in the violent-video-games cluster share.
5. This happens everywhere, but this field especially is prone to journalists misreporting and misinterpreting the literature.