Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Numerai – A hedge fund built by a global community of anonymous data scientists

numer.ai

Post reply on HN
Print view View original

51–60 of 120 posts

Re: Numerai – A hedge fund built by a global community of anonymous data scientists

#51

Post by fixxer »

lordnacho wrote: post #16

Hedge fund guy here. - You need to know something about the domain in order to make sensible predictions. Is the data daily? Is it per second? Is it ticks? You can't build a sensible model if you don't know that, even if you have good predictions. Relative cost will vary a lot between timescales. - It matters what the features are. Maybe there's some clever reason why it doesn't, but until I hear why I'm going to tak…

Based on the contest (weekly submission) and data (binary classification), it is my guess that they are running a long-short equity portfolio with weekly rebalancing. The features are known ahead of time and the dataset is not BIG, so I think some sort of fundamental approach.

Im not optimistic...

Re: Numerai – A hedge fund built by a global community of anonymous data scientists

#53

Post by jamez1 »

ktamura wrote: post #37

Their dataset reeks of startup hustling. I just downloaded the training set [1] and plotted some of its descriptive statistics [2]. It looks that all features are uniform distributions and the response variable is Bernoulli coin-flipping. In layman's terms, you can't really come up with a good predictive model with this training set. I give them the benefit of the doubt that they wanted to have something in place to…

The key property that you should be testing for is dependence. It would be surprising if you could identify anything in the descriptive statistics, that would make this all child's play?

I can't imagine any serious data scientist not knowing that.

Re: Numerai – A hedge fund built by a global community of anonymous data scientists

#54

Post by dooglius »

This is fishy. The entire point of encrypted data is that one message cannot be distinguished from another without decryption (which would require the private key). In other words, the entire premise shouldn't work. This means that either bad encryption is being used (i.e. statistical information about the data is leaked) or the good results we see are just noise. Or, the whole thing is a scam to get funding: the best algorithms are planted, and the company just shuffles BTC between accounts it controls.

Re: Numerai – A hedge fund built by a global community of anonymous data scientists

#55

Post by asdfologist »

dreamdu5t wrote: post #44
asdfologist wrote: post #39

I don't see how Numerai can avoid the multiple comparisons problem [0]. If people submit thousands of random models, then some subset of them will do a fantastic job in predicting prices in historical simulations but do poorly under real market conditions. As long as the models are black boxes, there's likely no good way to distinguish them from noise. [0] https://en.wikipedia.org/wiki/Multiple_comparisons_problem

That's true only if stock market data is completely random and that there's no signal to predict on. That doesn't seem to be the case, considering hedge funds successfully use ML on historical data to capture alpha. You don't need a model to work forever to be successful.

It looks like you missed the point of my comment. I'm saying that Numerai won't be able to distinguish between zero alpha and positive alpha models, if all they're doing is running historical simulations on black boxes.

Re: Numerai – A hedge fund built by a global community of anonymous data scientists

#56

Post by jamez1 »

bigdubs wrote: post #6
kowdermeister wrote: post #5

"In December 2015, we created the world’s first encrypted data science tournament for stock market predictions. Since then, Numerai data scientists have submitted 13,350,675,598 equity price predictions. The most accurate and original machine learning models from the world’s best data scientists are synthesized into a collective artificial intelligence that controls the capital in Numerai’s hedge fund." What does thi…

Monkeys throwing darts at the board. This brings to mind the Buffett hedge fund wager, where he invested in a vanguard s&p 500 tracking fund (VFIAX) and a hedge fund actively managed an equal amount, and Mr. Buffett ended up winning handily.

Buffet's wager is a trick, which works on simple people who don't understand statistics.

When you take a group of hedge funds, they end up being a very good proxy for the market. At about 20, they will be almost indiscernible.

So - a passive fund, buying the market, with lower fees, should outperform.

Re: Numerai – A hedge fund built by a global community of anonymous data scientists

#57

Post by [deleted] »

lordnacho wrote: post #16

Hedge fund guy here. - You need to know something about the domain in order to make sensible predictions. Is the data daily? Is it per second? Is it ticks? You can't build a sensible model if you don't know that, even if you have good predictions. Relative cost will vary a lot between timescales. - It matters what the features are. Maybe there's some clever reason why it doesn't, but until I hear why I'm going to tak…

[deleted]

Re: Numerai – A hedge fund built by a global community of anonymous data scientists

#58

Post by jamez1 »

lordnacho wrote: post #16

Hedge fund guy here. - You need to know something about the domain in order to make sensible predictions. Is the data daily? Is it per second? Is it ticks? You can't build a sensible model if you don't know that, even if you have good predictions. Relative cost will vary a lot between timescales. - It matters what the features are. Maybe there's some clever reason why it doesn't, but until I hear why I'm going to tak…

Your constraints are rather simplistic given the people behind this. It is possible, that all of the factors you've just described, are encoded in the data set.

Re: Numerai – A hedge fund built by a global community of anonymous data scientists

#59

Post by jamez1 »

dooglius wrote: post #54

This is fishy. The entire point of encrypted data is that one message cannot be distinguished from another without decryption (which would require the private key). In other words, the entire premise shouldn't work. This means that either bad encryption is being used (i.e. statistical information about the data is leaked) or the good results we see are just noise. Or, the whole thing is a scam to get funding: the bes…

The data is homomorphically encrypted, meaning you can do operations (such as add and subtract) on the ciphered message and it will also perform them on the underlying data.

Re: Numerai – A hedge fund built by a global community of anonymous data scientists

#60

Post by dooglius »

jamez1 wrote: post #59
dooglius wrote: post #54

This is fishy. The entire point of encrypted data is that one message cannot be distinguished from another without decryption (which would require the private key). In other words, the entire premise shouldn't work. This means that either bad encryption is being used (i.e. statistical information about the data is leaked) or the good results we see are just noise. Or, the whole thing is a scam to get funding: the bes…

The data is homomorphically encrypted, meaning you can do operations (such as add and subtract) on the ciphered message and it will also perform them on the underlying data.

Yes, I realize that. The issue is that the result of any operation is also encrypted, which means that there should be no way to connect the target of the training data (encrypted or not) to the output of a function of encrypted data. Suppose the unencrypted data is (a,b,c) where a+b=c, and (x,y,z)=encrypt((a,b,c)). We have an addition function plus on encrypted data such that decrypt(plus(x,y))=a+b=c=decrypt(z), but it is not the case that plus(x,y)=z (at least, not if plus is computable in polynomial time, and assuming the encryption scheme is sound). If it were, we could statistically distinguish encrypt((a,b,c)) from encrypt((rand(),rand(),rand())) which would mean the encryption is not sound.
Post reply on HN