Many people used to have knee arthroscopy. Doctors felt it was useful. Patients thought it was useful. It had a credible mechanism of action.
When we do double blind placebo controlled studies we find that knee arthroscopy for some conditions is no better than placebo.
> Why is it okay to report that Joe said "I feel less depressed when I exercise" or "I feel less depressed when I get to bed earlier" but NOT okay to report that Joe said "I feel less depressed when I use the tCDS"?
It isn't okay to tell people to do exercise to treat their depression unless you have some kind of evidence to back that up. The evidence is weak, and the effect is small, but more research needs to be done. (http://summaries.cochrane.org/CD004366/exercise-for-depressi...)
It is not okay to tell people to go to bed earlier to treat their depression unless you have evidence to back that up. I can think of a bunch of pros and cons for going to be earlier. I wouldn't voice my uninformed opinion in a national newspaper, and I wouldn't want that national newspaper to trumpet my uninformed opinion as a facty truthy comment.
> Why is he allowed to KNOW those first two things based just on personal knowledge of how he feels, but not that last one?
My position is that he doesn't know any of those things, not that he can know two of them but not the last one. Especially things like your gin / vodka example: I'd want evidence. Get a bunch of people in a room. Dip the rim of some drinking glasses in gin, and fill the glasses with cold tonic water. Observe. They're not drinking any alcohol, but they do have loosened inhibitions.
So far I've just spoken about the need for a trial. Good quality trials are always needed. Cognitive biases are very strong, and very hard to protect against, and can cause people to undergo lengthy, risky, poor quality "treatment" that just doesn't work. (There are many good books about this. "Bad Science" is a good quick read.)
Let's look at some other flags that might cause us to want better data, rather than just accept his word.
He sells the device for $600. Right there is a big flag.
Either he knows it doesn't work (I don't think this is the case) and he's a scumbag, cynically exploiting people. Or he really does think it works, and he wants to help people. Either way, better data is a good thing. Better data helps us stop the scumbags, and better data helps people like him sell the device and treat ill people.