Do Men Suck At Friendship?

However, my objection is that I don't believe language can be easily classified in terms of the response it'll elicit. Obviously, one can (usually) correctly guess the response that'll be received if one were to run up to a stranger and yell "You're a [swearword of choice here]", yet the fact that I've had to preface this with the modifier 'usually' betrays my point; some people will get aggressive if you swear at them, some will laugh, some will respond in kind, and so on. My concern is that if we can't even predict the effect of language in its most obvious state, we probably can't predict its effect in subtler states.

This unpredictability of language leaves us in a tricky position when it comes to asking questions on a self-reporting study. In order to solve that one objection, we'd have to come up with a method of using language which manages to communicate its point, without causing that point to make people feel emotion. This is further complicated by the fact that people are complex beasts with internal and external factors playing in to how they behave, such that a question formed neutrally for one person would probably not be so neutral with others. This also makes avoiding 'fear of reprisal' for one's response to a question impossible, as we can only remove external reprisal. It would not be possible for us to, for instance, removal the internal upheaval of a conflicted homosexual admitting to a survey that they were gay.

See, you used the term 'suggests' rather than 'proves' because you know that claiming a stronger relationship between upvotes and motive would be affirming the consequent. But this is precisely the sort of weasel-wording which I've seen in observational studies, and it seems deliberately crafted to trick an uncanny reader ill-versed in logic into misinterpreting 'suggests' as 'proves'. Of course, we both know that we cannot infer anything from a consequent other than one of the possible antecedents must have occurred, and we both know that the antecedents in this case -- motive for clicking upvote -- is huge, and thus nothing meaningful can be inferred about the consequent. I'm happy to have a discussion about almost anything, but if someone comes to the party with nonsense evidence pretending the discussion has already been studied and decided, I'm going to call them on it.

I also feel like you've also dodged every point I've raised (or perhaps I didn't explain my objections very well). With regards to #1, the issue wasn't that I think researchers are deliberately crafting leading questions, but that in order for the study to be valid they'd have to show that their questions either do not lead thus aren't confounding (which I've argued is impossible), or that they lead predictably thus can be countered in the analysis (which I also argued is impossible).

With #2 you're correct that this is an issue for all studies, but it's a particularly large issue for studies of things which are irreducibly complex, like people. Since we can't (easily) take specific facets of a person and study those in silo from the rest of a person, controlling confounding variables becomes a bigger issue. Even in other observational sciences we can usually demonstrate the core parts of our assumptions in a controlled experimental manner. For instance, in the study of global warming, we can demonstrate in a controlled, experimental way that the combustion of fossil fuels releases CO2. With studies of human behaviour this is rarely possible.

With #3 you're correct that the media is far guiltier of this than the scientists, but I'd argue that scientists need to be more vocal about this issue. I appreciate this treads a fine line between asking for more scientific social responsibility, and holding scientists responsible for the behaviour of society, but I feel this is a valid concern due to the way that politicians like to fund studies such as these to validate their personal opinion. The reason I believe this important isn't that I think scientists are trying to dupe us, far from it, but because it worries me that as burden the of proof for a posteriori logic falls from the strongly codified and philosophically justified rules of empiricism and falsifiability, so scientists move from being discoverers of truth to yet another controllable authority figure.

Also, thank you again for citing evidence for your point. I apologise that I have not done so, but I seriously doubt any scientists actually agree with me here. Having read your linked study, I would say it both stands to reason and doesn't really seem to prove the point it claims to prove. If you set out to prove that self-reporting isn't invalidated by confounding variables, and you do so by invoking self-reporting which contains almost exactly the same confounding variables, then you can't really claim to have proved anything. Relatives and friends of a sample in such a study would be just as likely to change their answers, consciously or subconsciously, to avoid internal conflict, and because they're tied to the sample in such a way that would produce a similar personality and similar self-identity reprisals if the subject's life choice were cast in doubt, it's also not a large leap of logic that their changed answers would usually change along the same lines as the sample.

Again, I can't really think of a better way of studying complex issues like human behaviour, but since we started at the point of 'science agrees self-reporting is fine' and are now at 'we agree it is the best we can get', I feel we're moving in the right direction. I do agree that well-controlled self-report studies are probably the best we can get in this field, it just seems to me that the best we can get isn't as valid as the best we can get in experimental sciences, and should be noted as such.