Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Numerai – A hedge fund built by a global community of anonymous data scientists

numer.ai

Post reply on HN
Print view View original

41–50 of 120 posts

Re: Numerai – A hedge fund built by a global community of anonymous data scientists

#41

Post by mikbob »

jon_richards wrote: post #20
gravypod wrote: post #11

Couldn't I make thousands of fake accounts and submit thousands of slightly different models. Then after a while I could push one or two of my stocks higher in my models artificially. If you represented a large enough % of the "data scientists" in this hedge fund you could make it look like your stock is "definitely" work investment. After they invest in your company, you could take off to the hills. Hell, you don't…

If they aren't managing correlation between different models there is an even bigger issue...

They are - your rank on the leaderboard is the importance of your model in their 'meta-model' so submitting the same model as someone else would just net you 0.

Re: Numerai – A hedge fund built by a global community of anonymous data scientists

#42

Post by mikbob »

ikeboy wrote: post #23
dsacco wrote: post #9

Earlier quoted context omitted.

Leaving out the marketing hyperbole, it means that Numerai's core software is an automated trading system derived from, and ostensibly trained on, the most accurate pricing models and insights put forward by participants (the data scientists). Participants with the most accurate [1] insights are paid a sum in exchange for the insights being incorporated into the hedge fund's trading system, which in turn manages the…

The "predictions" are submitted only knowing a portion of the dataset, but the winners are chosen based on the entire dataset, which is supposed to prevent overfitting.

Winners are chosen based on a part of the test dataset which the user never gets feedback on/never is able to see the true values of during the competition round - so it's very difficult to overfit to that private set.

Re: Numerai – A hedge fund built by a global community of anonymous data scientists

#43

Post by noobermin »

gravypod wrote: post #11

Couldn't I make thousands of fake accounts and submit thousands of slightly different models. Then after a while I could push one or two of my stocks higher in my models artificially. If you represented a large enough % of the "data scientists" in this hedge fund you could make it look like your stock is "definitely" work investment. After they invest in your company, you could take off to the hills. Hell, you don't…

Shhh! Don't give my strategy away!

Re: Numerai – A hedge fund built by a global community of anonymous data scientists

#44

Post by dreamdu5t »

asdfologist wrote: post #39

I don't see how Numerai can avoid the multiple comparisons problem [0]. If people submit thousands of random models, then some subset of them will do a fantastic job in predicting prices in historical simulations but do poorly under real market conditions. As long as the models are black boxes, there's likely no good way to distinguish them from noise. [0] https://en.wikipedia.org/wiki/Multiple_comparisons_problem

That's true only if stock market data is completely random and that there's no signal to predict on. That doesn't seem to be the case, considering hedge funds successfully use ML on historical data to capture alpha. You don't need a model to work forever to be successful.

Re: Numerai – A hedge fund built by a global community of anonymous data scientists

#45

Post by dschiptsov »

Someone got financing for a crowdsourcing of data mining of commercial data set. Very clever.

It seems that only very few nerds are taking the theoretical impossibility of predicting the future seriously and we are missing the opportunity to get funding for some crappy models from the greater fools.

Re: Numerai – A hedge fund built by a global community of anonymous data scientists

#46

Post by daveguy »

antognini wrote: post #38
ktamura wrote: post #37

Their dataset reeks of startup hustling. I just downloaded the training set [1] and plotted some of its descriptive statistics [2]. It looks that all features are uniform distributions and the response variable is Bernoulli coin-flipping. In layman's terms, you can't really come up with a good predictive model with this training set. I give them the benefit of the doubt that they wanted to have something in place to…

The features are all encrypted using homomorphic encryption, which ends up mapping the overall distribuiton uniformly onto [0, 1]. If you play with the dataset, though, you'll find there are some significant correlations between the different features and you can make a model that does substantially better than chance.

I would like to see a comparison between random generated uniform features and these features. Can you fit the noise and come up with "statistically significant" predictions? If so, is anyone doing any better than can be done on random data? If no one is doing better than that, what is the likelihood that these aren't any better than monkey with a dartboard?

I would love to see peer reviewed articles from numerai with some of their behind the scenes results.

Re: Numerai – A hedge fund built by a global community of anonymous data scientists

#47

Post by Houshalter »

I don't understand how this works. What exactly is being predicted? The data isn't a time series. The outputs are only binary. There are only 27 features. I don't understand how this represents market data at all. In fact they probably destroyed most of the information trying to convert the data to this format.

Re: Numerai – A hedge fund built by a global community of anonymous data scientists

#48

Post by Houshalter »

gravypod wrote: post #11

Couldn't I make thousands of fake accounts and submit thousands of slightly different models. Then after a while I could push one or two of my stocks higher in my models artificially. If you represented a large enough % of the "data scientists" in this hedge fund you could make it look like your stock is "definitely" work investment. After they invest in your company, you could take off to the hills. Hell, you don't…

This won't work. As far as I can tell, you don't get to "pick stocks". You just submit predictions for unlabelled variables that could represent anything. It's no different than a standard kaggle competition.

Re: Numerai – A hedge fund built by a global community of anonymous data scientists

#49

Post by wodenokoto »

bigdubs wrote: post #6
kowdermeister wrote: post #5

"In December 2015, we created the world’s first encrypted data science tournament for stock market predictions. Since then, Numerai data scientists have submitted 13,350,675,598 equity price predictions. The most accurate and original machine learning models from the world’s best data scientists are synthesized into a collective artificial intelligence that controls the capital in Numerai’s hedge fund." What does thi…

Monkeys throwing darts at the board. This brings to mind the Buffett hedge fund wager, where he invested in a vanguard s&p 500 tracking fund (VFIAX) and a hedge fund actively managed an equal amount, and Mr. Buffett ended up winning handily.

Investing in S&P500 is a strategy, which can be optimised. You can ask if it is better to invest in top 100 or top 1000 instead of a top 500 index. Why even go after the top index? There are all sorts of inactive strategies available.

So in a sense, the "active" manager is just as much a monkey throwing darts as the "inactive", yet one of them is better than the other.

Why can't a third monkey be even better?

Re: Numerai – A hedge fund built by a global community of anonymous data scientists

#50

Post by daveguy »

zitterbewegung wrote: post #3

Why should I think that being anonymous should give some advantage ? I think it would be more of a disadvantage due to should I trust you? Also, how do I know these people are truly anonymous ?

Anonymous in the sense that numerai does not ask for any personally identifying information. You could easily prove you are who you say you are in numerai.

Also, I think the big benefit from a data scientist working on this is you can test methods with generic features, submit the results and get paid if they are good, but not submit any part of the methodology to any third party.

If you can kill it on numerai then maybe you would consider buying data sources and apply your methods to your own data. Although you still don't know what the features are.

It's the polar opposite of open source.

The owners don't have to trust the data scientists. They evaluate their results against additional data.

Post reply on HN