Earlier quoted context omitted.
I'm sorry, but that's a terrible comment. Medical progress often starts with observations, and is confirmed by trials. Penicillin is one such example. To discount the value of observations is to ignore an important part of the discovery process. There is extremely limited data available around the use of nootropics for mind enhancement (as opposed to in medical use for treating illness). In the absence of of such dat…
Observations are one thing, attempts to do any kind of serious study on the results are another. It is not meaningful to do statistics on data that bad.
We can safely report the statistic that "more people self report caffeine having an impact than ginseng."
The author goes as far as noting how many people are reporting an impact where there appears to be no known mechanism that impact could be occurring. That's a defensible statistic too.
Data from self-reported surveys isn't entirely useless, and is in medical research often enough for there to be studies on the kinds of error that arise from them: http://scholar.google.com.au/scholar?q=self+reporting+survey...