[deleted]
Numerai, a hedge fund built by a community of anonymous data scientists
21–30 of 84 posts
Re: Numerai, a hedge fund built by a community of anonymous data scientists
#22I'm skeptical. There are skyscrapers in NYC's, Londons, Singapores and Hong Kong's of this world filled with people who are smart and have enormous computer resources and funds and are paid handsomely to work on solving this problem with all manners of ML and AI at their disposal, the "crowd" has no advantage over them. The "closed system" is much larger than the "crowd" in this case.
Re: Numerai, a hedge fund built by a community of anonymous data scientists
#23"It is intuitively obvious that an open access hedge fund will generate more intelligence than a closed system built on a pre-internet, pre-cryptocurrency, pre-AI organizational design." Really? Because the folks with the magic black box aren't capable of funding an Interactive Brokers account to keep 100% of their upside and 100% of their IP? (Also: risk management and order handling are harder problems than signal…
Isn't that the magic of OSS? Linux::Windows, matplotlib::mathematica, android::iPhone, etc. In each case, the free variety quickly catches up to the proprietary version, and in doing so, cuts into the profitability of the parent. Furthermore, this often breaks down monopolies, as they must innovate or die.
Re: Numerai, a hedge fund built by a community of anonymous data scientists
#24Earlier quoted context omitted.
Can you clarify on the "pay tiny prices and shut the competitions once they have found what they were looking for" comment? Has this happened before? Is there anything here that makes it seem like this wouldn't be the case here?
They only require you to give them predictions, not a model. So you can reuse the same model over and over if it works, and they won't be able to cut you out, if that's what you're asking.
That was part of what I was asking. The other part was around whether there have been instances of firms cutting people out like you described.
Re: Numerai, a hedge fund built by a community of anonymous data scientists
#25"It is intuitively obvious that an open access hedge fund will generate more intelligence than a closed system built on a pre-internet, pre-cryptocurrency, pre-AI organizational design." Really? Because the folks with the magic black box aren't capable of funding an Interactive Brokers account to keep 100% of their upside and 100% of their IP? (Also: risk management and order handling are harder problems than signal…
Isn't that the magic of OSS? Linux::Windows, matplotlib::mathematica, android::iPhone, etc. In each case, the free variety quickly catches up to the proprietary version, and in doing so, cuts into the profitability of the parent. Furthermore, this often breaks down monopolies, as they must innovate or die.
Matplotlib covers a tiny spec of a footnote of Mathematica's functionality. SageMath (a composition of Numpy, Scipy, Sympy, matplotlib, R, etc.) is a more appropriate analogy.
Re: Numerai, a hedge fund built by a community of anonymous data scientists
#26Earlier quoted context omitted.
If the machine is "anonymous", it does matter. Scenario #1: A human gets insider information that a Solar City will be bought. Makes his anonymous "AI machine" predict that Solar City is a great stock to buy!
That's not possible. The data is encrypted. None of the participants can see which stocks (or anything) about the data they train with. Numerai turns stock prediction into a pure ML problem.
Okay, submit hundreds of bots, and have them generate random picks. One gets lucky, and is selected for hilighting.
Re: Numerai, a hedge fund built by a community of anonymous data scientists
#27Understanding which features to create and why is significantly more impactful than just trying new models on the same dataset.
Re: Numerai, a hedge fund built by a community of anonymous data scientists
#28I started poking at this out of curiosity, and a desire to begin sharpening my TensorFlow axe, and one thing remains unclear. They give you two spreadsheets, one being the training data and the other is the tournament data (what you need to predict on). Each entry in the spreadsheet is 21 features and a single binary class. The latter is what you predict. But for the submissions they request a probability, not a class. They don't explain what "probability" here means. Does it mean probability of class 0? Probability of class 1? Probability of the moon exploding on a Thursday?
Overall interesting idea. Undecided whether it's real/scam/fake, but definitely very interesting at face value. I just wish their documentation was more clear. Seems kind of important...
EDIT: Found a comment on Reddit that indicates that it means probability of class 1 (https://www.reddit.com/r/MachineLearning/comments/3wdr9e/num...)
Re: Numerai, a hedge fund built by a community of anonymous data scientists
#29Earlier quoted context omitted.
If the machine is "anonymous", it does matter. Scenario #1: A human gets insider information that a Solar City will be bought. Makes his anonymous "AI machine" predict that Solar City is a great stock to buy!
That's not possible. The data is encrypted. None of the participants can see which stocks (or anything) about the data they train with. Numerai turns stock prediction into a pure ML problem.
So when your "AI" makes a recommendation, it recommends some encrypted symbol?
Re: Numerai, a hedge fund built by a community of anonymous data scientists
#30I'm skeptical. There are skyscrapers in NYC's, Londons, Singapores and Hong Kong's of this world filled with people who are smart and have enormous computer resources and funds and are paid handsomely to work on solving this problem with all manners of ML and AI at their disposal, the "crowd" has no advantage over them. The "closed system" is much larger than the "crowd" in this case.
I don't think this is true at all. 10,000 people are just going to have more ideas and better individual ideas than 100 experts. The impact of that much creativity and perspective can be exponential, and it's hard to duplicate.
When I'm designing a system, I hate to have to try to out think everyone on the internet. If you have a known set of opponents you can predict what they might do. When you're up against anybody from anywhere, you never know what you're going to get. Global scale collaboration is a very powerful thing because it allows a complete exploration of the solution space, and it's difficult to stop.