MIT computer scientists can predict the price of Bitcoin • Hacker News Board

Re: MIT computer scientists can predict the price of Bitcoin

#81

Post by Tloewald » Sat, Oct 25, 2014, 2:16 PM UTC

panarky wrote: post #34

lrm242 wrote: post #26

Earlier quoted context omitted.

You're mixing the concepts of realized and un-realized profit. A market price change marks your open position and gives you an unrealized (loss)gain. But to realize that you have to trade, plain and simple. And someone must take the other side of that. A more compelling argument against zero-sum is that traders and investors have different time frames and objectives. And, indeed, the classic argument in the futures m…

I start a company to make cheap autonomous flying cars. I invest $1,000 of my own money and build a working prototype. It looks promising, so you buy 50% of my shares for $10,000. I just made a 20x return on my investment, and you own 50% of a great opportunity. Who's the loser here that makes this a zero-sum market? Our autonomous flying cars go into production, and now your 50% is worth $100 billion, and you sell y…

Once your company has shares, you made $10,000 for 50% of your stock which you paid $500 for. But the investor merely has your shares. So you are up $9000, whoever got your $1000 is up $1000, and the investor is down $10,000. That adds up to zero.

Again, not saying you're wrong, but your example is useless.

Re: MIT computer scientists can predict the price of Bitcoin

#82

Post by puredemo » Sat, Oct 25, 2014, 2:41 PM UTC

Houshalter wrote: post #48

Sometime in 2013, before the bitcoin prices exploded, I downloaded some bitcoin historical price data and ran symbolic regression on it with Eureqa. It came up with a formula that fit the observed data fairly well, and wasn't very complicated. But when I extrapolated it forward a few months, it predicted the price would explode to unreasonable levels. I was disappointed and threw it away, assuming that it must be wro…

What does it say when you run it now?

Re: MIT computer scientists can predict the price of Bitcoin

#83

Post by beejiu » Sat, Oct 25, 2014, 4:06 PM UTC

leeber wrote: post #51

barisser wrote: post #4

This seems like massive historical overfit, which can lead to arbitrarily precise fit, but no predictive capability. Any model, if given enough parameters, can be made to match historical data to an arbitrary degree. I also run several Bitcoin bots. I can tell you that slippage is not insignificant. If you make transactions every ~10 seconds and incur 0.1% fees each time, this is an extremely significant effect in ag…

Agreed. According to the paper: They trained the data, once, with 3-4 months of data from Feb to May. Then they tested the data, once, with ~6 weeks of data from May 6 - June 24. There was no cross validation involved in assessing the performance of this model. Using one subset for training, and one subset for testing and calling that conclusive is naive. No interesting conclusions can be taken away from this paper d…

Cross validation too uses training sets and test sets. This sort of time-ordered data will not be independent, so the prequential approach seems to be a more suitable approach to measuring forecast accuracy. (I haven't read the paper.)

Re: MIT computer scientists can predict the price of Bitcoin

#84

Post by leeber » Sat, Oct 25, 2014, 6:00 PM UTC

beejiu wrote: post #83

leeber wrote: post #51

Earlier quoted context omitted.

Agreed. According to the paper: They trained the data, once, with 3-4 months of data from Feb to May. Then they tested the data, once, with ~6 weeks of data from May 6 - June 24. There was no cross validation involved in assessing the performance of this model. Using one subset for training, and one subset for testing and calling that conclusive is naive. No interesting conclusions can be taken away from this paper d…

Cross validation too uses training sets and test sets. This sort of time-ordered data will not be independent, so the prequential approach seems to be a more suitable approach to measuring forecast accuracy. (I haven't read the paper.)

Cross validation for time series, goes something like this:

http://robjhyndman.com/hyndsight/crossvalidation/

(near bottom of the article)

Anyways, as somebody who has spent GIANT amounts of time experimenting with machine learning using market data (mainly stocks), I can give you TONS algorithms I've created that would show similar, even much higher, gains when you only test on 6 weeks of data. Been there done that. For example 6 weeks you make a 100% return, then in the next 3 weeks you suffer a 50% loss. Reality sets in...

Re: MIT computer scientists can predict the price of Bitcoin

#85

Post by dratman » Sun, Oct 26, 2014, 4:08 AM UTC

api wrote: post #9

barisser wrote: post #4

This seems like massive historical overfit, which can lead to arbitrarily precise fit, but no predictive capability. Any model, if given enough parameters, can be made to match historical data to an arbitrary degree. I also run several Bitcoin bots. I can tell you that slippage is not insignificant. If you make transactions every ~10 seconds and incur 0.1% fees each time, this is an extremely significant effect in ag…

"Any model, if given enough parameters, can be made to match historical data to an arbitrary degree." I've known this forever but for some reason haven't heard this precise statement of it. Thanks. Reductio ad absurdium: imagine a model where the number of parameters equals the number of data points. Obviously that model will have perfect fit. Predicting the future is hard. Predicting the future without a causal unde…

"Predicting the future without a causal understanding of the system is epistemologically questionable."

That is an intriguing assertion, but it is circular. One can only demonstrate a causal understanding of a system by making usefully accurate predictions about the system's behavior.

To attempt that with a system consisting of market prices of tradeable securities is an exercise in frustration, because such markets do not operate by consistent, unchanging causal rules. In fact, financial markets are not systems at all in the usual sense, because their parts and connections are continually changing.

Re: MIT computer scientists can predict the price of Bitcoin

#86

Post by hft_throwaway » Sun, Oct 26, 2014, 6:17 PM UTC

zaptheimpaler wrote: post #80

minimax wrote: post #29

The problem with the paper is not overfit. They claim to have run their simulation with out of band ("live") data. The actual problem with the paper is that we have no idea if their simulator is any good, which means that their result (89% return in 50 days) could be totally bogus. In other words, we don't know if the actual bitcoin exchange would fill their orders at the same prices (if at all) as their simulator do…

From the article: >Specifically, every two seconds they predicted the average price movement over the following 10 seconds. If the price movement was higher than a certain threshold, they bought a Bitcoin; if it was lower than the opposite threshold, they sold one; and if it was in-between, they did nothing. If they are indeed predicting the price at X+10 seconds at second X, they have more than enough time to act on…

That assumes there is no bias to their signal returns over time, which is extremely unlikely. 90% of the moves that go their way could happen in the first second, for example.

Many HFTs hold positions for seconds or minutes, and predict prices out over similar time horizons, but they'll lose their best winners to competitors if they aren't fast and adept at executing.