Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

MIT computer scientists can predict the price of Bitcoin

newsoffice.mit.edu

Post reply on HN
Print view View original

51–60 of 86 posts

Re: MIT computer scientists can predict the price of Bitcoin

#51

Post by leeber »

barisser wrote: post #4

This seems like massive historical overfit, which can lead to arbitrarily precise fit, but no predictive capability. Any model, if given enough parameters, can be made to match historical data to an arbitrary degree. I also run several Bitcoin bots. I can tell you that slippage is not insignificant. If you make transactions every ~10 seconds and incur 0.1% fees each time, this is an extremely significant effect in ag…

Agreed. According to the paper: They trained the data, once, with 3-4 months of data from Feb to May. Then they tested the data, once, with ~6 weeks of data from May 6 - June 24.

There was no cross validation involved in assessing the performance of this model. Using one subset for training, and one subset for testing and calling that conclusive is naive.

No interesting conclusions can be taken away from this paper due to flawed methodology and failure to take into account real world variables like bid-ask spread, commissions, how quickly trades can be executed, whether orders will even be filled, etc.

Re: MIT computer scientists can predict the price of Bitcoin

#52

Post by Houshalter »

andrea_s wrote: post #20

How many of the people who are bashing the paper in this discussion are machine learning experts? Especially the overfitting crowd - looks like emotional attachment to one's favorite topics is not really impacted by said person's overall education and expertise.

Here is the discussion on /r/MachineLearning: https://www.reddit.com/r/MachineLearning/comments/2jwpv1/mit...

Re: MIT computer scientists can predict the price of Bitcoin

#53

Post by rational-future »

minimax wrote: post #29

The problem with the paper is not overfit. They claim to have run their simulation with out of band ("live") data. The actual problem with the paper is that we have no idea if their simulator is any good, which means that their result (89% return in 50 days) could be totally bogus. In other words, we don't know if the actual bitcoin exchange would fill their orders at the same prices (if at all) as their simulator do…

These guys did it right. You first build a simulator and publish a paper and then join a hedge fund for $$$$$$ and actually try the algo.

Re: MIT computer scientists can predict the price of Bitcoin

#54

Post by runeks »

Success in predicting markets is measured in profit.

This research team should start a company that offers a service that allows users to deposit bitcoins, which the company then invests according to their alleged predictions, and then pay interest on deposits, and keep a part of the profit for themselves.

Doubling the initial investment one time is one thing, but this hypothetical company being able to double its investment every 50 days for years is something else. I doubt they can. A doubling every 50 days is x160 every year.

I think claims of being able to predict market prices should be met with great skepticism. Especially prices of easily traded commodities, including bitcoins.

The only proper measure of an ability to predict market prices is profit, because profit also measures the extent of the predictions: how much can you move the market (by trading according to your predictions) until you can no longer predict what will happen? Obviously, there's a limit. No one can extract unlimited profit from any market. So there definitely is a limit to how much you can earn from your algorithm. If you can earn 10% p.a. on an investment of maximum $5000, your algorithm isn't really worth much. If you can earn 1000% p.a. on an investment of up to $100M, your algorithm is great. But without knowing these figures we really only have a claim, seemingly a claim of them being able to make a lot of money, but choosing not to do so.

Re: MIT computer scientists can predict the price of Bitcoin

#55

Post by [deleted] »

runeks wrote: post #54

Success in predicting markets is measured in profit. This research team should start a company that offers a service that allows users to deposit bitcoins, which the company then invests according to their alleged predictions, and then pay interest on deposits, and keep a part of the profit for themselves. Doubling the initial investment one time is one thing, but this hypothetical company being able to double its in…

[deleted]

Re: MIT computer scientists can predict the price of Bitcoin

#56

Post by ne0n »

dnautics wrote: post #46
jacobwcarlson wrote: post #2

The paper states that the strategy was simulated with live data and makes no mention of slippage. I've never traded bitcoin so I'm not sure how difficult it is to get fills, but that along with spreads are non-trivial components of real trading.

it would have been better if they set up an wallet and put bitcoin in it and demonstrated the trades that were automatically executed by this wallet. Since bitcoin is a ledger history, we could be 100% certain that SOME algorithm 'did the right thing', although we couldn't be certain that they spun up more than one algorithm and then just showed us the best one.

...That's not how Bitcoin trading works. In order to trade, your bitcoins need to be deposited into an exchange. Trades are not recorded on bitcoin's public ledger because there isn't a transaction for every trade. Bitcoin cannot currently handle that many transactions.

Re: MIT computer scientists can predict the price of Bitcoin

#58

Post by Dylan16807 »

minimax wrote: post #29

The problem with the paper is not overfit. They claim to have run their simulation with out of band ("live") data. The actual problem with the paper is that we have no idea if their simulator is any good, which means that their result (89% return in 50 days) could be totally bogus. In other words, we don't know if the actual bitcoin exchange would fill their orders at the same prices (if at all) as their simulator do…

I remember exchange delays on mtgox reaching into the minutes! This algorithm breaks with mere seconds of delay.

Re: MIT computer scientists can predict the price of Bitcoin

#59

Post by liaboc »

stokk wrote: post #57

Ever thought how our big data overlords (Google, Facebook, MS, Twitter, etc) just need to check for correlations between their users' data input and stock exchange movement? They own the stock "matrix".

You mean something like this?

http://financeai.com/stock/nyse/twtr

Re: MIT computer scientists can predict the price of Bitcoin

#60

Post by vasilipupkin »

leeber wrote: post #51
barisser wrote: post #4

This seems like massive historical overfit, which can lead to arbitrarily precise fit, but no predictive capability. Any model, if given enough parameters, can be made to match historical data to an arbitrary degree. I also run several Bitcoin bots. I can tell you that slippage is not insignificant. If you make transactions every ~10 seconds and incur 0.1% fees each time, this is an extremely significant effect in ag…

Agreed. According to the paper: They trained the data, once, with 3-4 months of data from Feb to May. Then they tested the data, once, with ~6 weeks of data from May 6 - June 24. There was no cross validation involved in assessing the performance of this model. Using one subset for training, and one subset for testing and calling that conclusive is naive. No interesting conclusions can be taken away from this paper d…

Well, they did trade it out of sample, so that probably means the model wasn't overfit. But, I agree, they probably used unrealistic execution assumptions
Post reply on HN