The Unbearable Asymmetry of Bullshit • Hacker News Board

Re: The Unbearable Asymmetry of Bullshit

#161

Post by worik » Tue, Mar 01, 2016, 8:09 PM UTC

xixi77 wrote: post #143

worik wrote: post #123

Earlier quoted context omitted.

No. That completely misrepresents the problem. tl;dr A false technique can be described and it can be hard or impossible to detect the technique is flawed by using it For example the paper at http://www.jstor.org/stable/222500 describes a method of using the stationary bootstrap to eliminate data snooping bias in studies of "technical analysis" in finance. Published in the Journal of Finance in 1999 at the time I wor…

Which exactly proof are you talking about? I briefly looked at the paper (I've seen it before, but it's been quite a while...), but it seems that they pretty much use a previously known approach, and the only proof in it is simply "replicated for convenience of the reader". Also, this would certainly be neither the first nor the last paper that ignores transaction costs, and their omission does not really invalidate…

Non-stationarity is enough to call bullshit. Really! How can a stationary bootstrap be used on data (financial returns) that are so prone to non-stationarity?

Irreducibility is also enough to call it very bad and should not have been published in that form. They talked a lot about their algorithms, without properly describing them.

Ignoring transaction costs is also enough to call bullshit. It is a mistake that should only be made by rank amateurs, and it is the most common mistake made by amateurs in the Technical Analysis field IMO.

It is a very very bad paper but because it gives a technique that can be used to show that TA is possible it is much beloved by researches in the field.

My own conclusion is that (generally) TA is not possible to do profitably at these time scales.

Re: The Unbearable Asymmetry of Bullshit

#162

Post by marvin » Tue, Mar 01, 2016, 10:17 PM UTC

liviu- wrote: post #157

marvin wrote: post #135

Earlier quoted context omitted.

A good Bachelors degree will prepare a good student for replicating science, and a good Masters degree will definitely leave a motivated and skilled student with a good advisor a master of his or her specific field. The problem is that grade inflation means the majority of students will fall short of these goalposts. I agree with your assessment that undergrad degrees represent hurdles, regardless of whether a studen…

> a good Masters degree will definitely leave a motivated and skilled student with a good advisor a master of his or her specific field. I'm having trouble believing this. In UK, most Masters degrees last 1 year, and there are several degrees considered good such as Imperial's MSc in Machine Learning, Cambridge's MPhil in Machine Learning, Speech and Language Technology, Edinburgh's MSc in Cognitive Science, and othe…

An MSc is not an MSc. It can commonly vary from 1-3 years, depending on institution. I was thinking of the two-year variety.

Of course, you can define the term "master" to mean pretty much whatever you like. But I'd say that two years of additional, focused study when you are already proficient in your field should be more than enough to have a mastery of the specific skills and knowledge that is at least on a high national level. I'm from Norway, so the US picture is unknown to me.

Re: The Unbearable Asymmetry of Bullshit

#163

Post by xixi77 » Thu, Mar 03, 2016, 5:37 PM UTC

worik wrote: post #161

xixi77 wrote: post #143

Earlier quoted context omitted.

Which exactly proof are you talking about? I briefly looked at the paper (I've seen it before, but it's been quite a while...), but it seems that they pretty much use a previously known approach, and the only proof in it is simply "replicated for convenience of the reader". Also, this would certainly be neither the first nor the last paper that ignores transaction costs, and their omission does not really invalidate…

Non-stationarity is enough to call bullshit. Really! How can a stationary bootstrap be used on data (financial returns) that are so prone to non-stationarity? Irreducibility is also enough to call it very bad and should not have been published in that form. They talked a lot about their algorithms, without properly describing them. Ignoring transaction costs is also enough to call bullshit. It is a mistake that shoul…

"Non-stationarity" is really an umbrella term; the truth is, there is no single authoritative model of asset returns, and really there will never be one. This should not preclude all statistical analysis though, and it is done by making simplifying assumptions, just like in every other case, and in every other field. Your claim is that they are too strong in this case, but it's a claim that can be fairly easily shown empirically or in simulations, and it really should be IMO, particularly since other bootstrap methods exist.

I agree about algorithms; rather unfortunately, this is true in more than this one paper. There certainly is movement towards requiring people to make their code fully available, but we are not quite there yet. But if you describe your failure to replicate, this is definitely a strong argument that the authors would IMO need to address.

Ignoring transaction costs would be a major problem if the paper's main point was "we found a strategy returning X% above market, it's awesome and people should give us money" -- but this is written for a very different purpose and audience. That being said, today it would not be published without a transaction cost analysis -- but I would have no problem with them saying "with such-and-such costs, profits are not there any more", it would not invalidate the paper at all. But at the time it was written, TC analysis was not as standard in academic literature as it is now.

I agree with you about TA, and TBH most serious researchers are of the same opinion, and have been for a long time -- even at the time of publication, it was a bit of an outlier, and this is not a particularly popular area of research (how many of those citations are in recent top journal articles?). Forex was a bit of an open question last time I checked, but it's been a few years, not sure if it still is.