A bot crawled thousands of studies looking for simple math errors

In Mathematics, if you get an unsolicited error report and the report is accurate, your first reaction is "oh my god" as you assess the damage and how much revision needs to be done to correct the paper. Then the answer is "thank you! I didn't realize so many people even read my work so closely! I'll add a correction and thank you for your contribution in an update of the paper".

neither are bullshit papers/editorials/letters-to-the-editor

http://frog.gatech.edu/Pubs/How-to-Publish-a-Scientific-Comm...

"Eminent" specialists often argue in favor of eminence (journals, tenured investigators, blah blah) over evidence (tossing an analysis up on arXiv and posting a link to it). Young turks or iconoclasts often do the reverse, since there is a power structure in place via editors, reviewers, study sections, and the like. Orthodoxy often sticks even when the evidence supporting it is quite skimpy (or absent).

Funny thing is, when you pull the underlying data (in fields where this is possible), you routinely find that the reported conclusions are overblown. Not necessarily wrong, but routinely sold as more conclusive than they are (and sometimes they are in fact wrong, whether due to small sample sizes, "outlier removal", or outright fraud).

In no way, shape, or form are these bad habits limited to psychology. They happen in basic biology, they happen in medicine, they happen in clinical trial reporting. It's faster and easier to oversell shitty science than it is to do good, thorough science, so with the tight competition for grant funding, you can imagine what happens next.

Trust whoever you like, but verify results from everyone. There are good people out there who are fastidious; there are good people out there who are sloppy; and there are people who don't care one bit whether they're publishing absolute horse shit. The onus is on you, the reader or researcher, to do the requisite critical thinking (and, perhaps, a few analyses before you waste time running down a dead end).

Keep an open mind, but don't let your brain fall out. Also, many good scientists aren't very nice people, and some very pleasant people in science are shitty scientists. It's very hard to tell a priori which is which, so look at the evidence and decide for yourself.

Because you're putting the cart before the horse. Science is building a picture of """objective reality""" from human subjectivity. Data points aren't reality, they have to be interpreted. No matter how you try to do it, at some point you have to accept that a human being is doing science and the way science is done is fundamentally an intersubjective phenomenon. Science isn't reality. Scientific knowledge isn't reality. Science is a human activity and scientific knowledge is a relationship between humans and reality. I'm not redefining what a fact is, I'm unpacking the definition of a scientific fact and scrutinizing its dynamics.

Why is this useful? Because if you don't look at science from this point of view, you can't understand or even recognize good or bad science. The scientific method can only fail in its execution. Its execution is fundamentally an intersubjective phenomenon.

There is a deeper philosophical advantage to this point of view: This is the correct formulation of science if you want to do science about science. That is, if you want to study scientifically the method by which discoveries are made, become verified, and evolve into commonly held "objective facts", you have to formulate science in this way. If you want to study scientifically how people become scientists, you have to formulate science this way. If you want to study why scientific illiteracy is still a major sociological problem, you have to formulate science this way. Et cetera.

In light of all this, I have to ask: What is the utility of the idea of objective facts, objective reality, a science that is not intersubjective, one that is merely acquired and applied in the masculine mode? My answer to that question is engineering: Engineering is not interested in doing science, but applying it. In which case, objective reality is merely a shortcut to using scientific knowledge. I think that's an admissible use, but it is inadequate for science itself and understanding the relationship between science and society.

P.S. When I say masculine and feminine, I certainly do not mean qualities intrinsic to men or women. I am referring to archetypal abstract masculinity and femininity which are themselves diffuse aspects of human intersubjectivity the study of which reveals long-scale correlations in human politics and social structure.

> Data points aren't reality, they have to be interpreted

Data points perhaps need to be interpreted for utility, but they represent facts (but those are facts of sense experience, which is ultimately subjective, though often expressed in terms of a fairly restrained conclusion of that subjective experience.)

Which is not to say I disagree with your broader point.

Generalizations are necessary for communication, cognition, general human understanding. If you try to reject them you will be rendered incapable of discussion, communication, thought about nearly any relevant or interesting issue.

Positions (1) (2) (3) are nearly axiomatically true -- if we are to generalize at all, and we must, we need to look for positions and people to generalize from. The very purpose of a professional organization is to aid us in doing this. Members of these organizations use it as a mental model to generalize "my professional community" and "the leaders of my professional community".

If you look at her resume, she is the former head of this and about 4 other high-prestige psychologists professional organizations, and a high ranking member of nearly every organization she is eligible to be a part of as a psychologist. She is the very definition as "well-respected representative of her field". She is a fine target for generalization. What you have not brought to the field are conflicting reports from other psychologists contradicting her or suggesting she is not an appropriate representative. Jimmy Carter and John McAfee are readily directly contradicted -- if you ask me to provide sources for numerous professionals in the field suggesting they are not the standard I can do so easily and so can you (though Jimmy Carter is a weird selection -- he is not a professional, and both are telling, since neither of them work in their field anymore)