Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Numerai, a hedge fund built by a community of anonymous data scientists

medium.com

Post reply on HN
Print view View original

11–20 of 84 posts

Re: Numerai, a hedge fund built by a community of anonymous data scientists

#11

Post by shostack »

davnn wrote: post #10

Interesting idea. Not that crowd driven investment algorithms are new, but I have not seen a machine learning one before. What really ennoys me about this kind of businesses is that they pay tiny prices and shut the competitions once they have found what they were looking for, however Numerai might be completely different in that regard and I wish them the best! Btw: The article kind of conveys the feeling as if mach…

Can you clarify on the "pay tiny prices and shut the competitions once they have found what they were looking for" comment? Has this happened before? Is there anything here that makes it seem like this wouldn't be the case here?

Re: Numerai, a hedge fund built by a community of anonymous data scientists

#12

Post by ikeboy »

theli0nheart wrote: post #8
alexmingoia wrote: post #7

Earlier quoted context omitted.

That's not possible. The data is encrypted. None of the participants can see which stocks (or anything) about the data they train with. Numerai turns stock prediction into a pure ML problem.

Doesn't the encrypted chart still need to display price history or volume? If so it seems like it'd be a trivial task to match it up with its real-world counterpart.

See https://en.wikipedia.org/wiki/Homomorphic_encryption, which they claim to use.

Re: Numerai, a hedge fund built by a community of anonymous data scientists

#13

Post by thedlade »

davnn wrote: post #10

Interesting idea. Not that crowd driven investment algorithms are new, but I have not seen a machine learning one before. What really ennoys me about this kind of businesses is that they pay tiny prices and shut the competitions once they have found what they were looking for, however Numerai might be completely different in that regard and I wish them the best! Btw: The article kind of conveys the feeling as if mach…

Exactly, machine learning has already been applied in a number of Wall Street firms for some time now

Re: Numerai, a hedge fund built by a community of anonymous data scientists

#14

Post by richard_craib »

davnn wrote: post #10

Interesting idea. Not that crowd driven investment algorithms are new, but I have not seen a machine learning one before. What really ennoys me about this kind of businesses is that they pay tiny prices and shut the competitions once they have found what they were looking for, however Numerai might be completely different in that regard and I wish them the best! Btw: The article kind of conveys the feeling as if mach…

Every other machine learning competition I've seen is kind of one-off in nature. But stock market data is being generated all the time so I think there will always be new strategies to learn on Numerai.

Re: Numerai, a hedge fund built by a community of anonymous data scientists

#15

Post by dharmon »

davnn wrote: post #10

Interesting idea. Not that crowd driven investment algorithms are new, but I have not seen a machine learning one before. What really ennoys me about this kind of businesses is that they pay tiny prices and shut the competitions once they have found what they were looking for, however Numerai might be completely different in that regard and I wish them the best! Btw: The article kind of conveys the feeling as if mach…

Even more than "a couple of years". About 15 years ago I worked at a day trading firm and we were writing models that used machine learning. At the time we thought of it more as "computational statistics", but its basically what is called ML now and taught in ML courses (although we didn't use Neural Nets).

BTW, even in 2001 we were far from the first to do this.

Re: Numerai, a hedge fund built by a community of anonymous data scientists

#16

Post by T-A »

dharmon wrote: post #15
davnn wrote: post #10

Interesting idea. Not that crowd driven investment algorithms are new, but I have not seen a machine learning one before. What really ennoys me about this kind of businesses is that they pay tiny prices and shut the competitions once they have found what they were looking for, however Numerai might be completely different in that regard and I wish them the best! Btw: The article kind of conveys the feeling as if mach…

Even more than "a couple of years". About 15 years ago I worked at a day trading firm and we were writing models that used machine learning. At the time we thought of it more as "computational statistics", but its basically what is called ML now and taught in ML courses (although we didn't use Neural Nets). BTW, even in 2001 we were far from the first to do this.

Depending on where you draw the line between statistics and machine learning, it could be argued to have originated with Bachelier's thesis in 1900 [1], or Thorp's adaptation of Kelly's work first to gambling and then to finance in the early 60s (he may have been first to use computers for this kind of thing) [2] or maybe with James Simons' Renaissance Technologies in the early 80s [3].

[1] https://en.wikipedia.org/wiki/Louis_Bachelier

[2] https://en.wikipedia.org/wiki/Edward_O._Thorp

[3] https://en.wikipedia.org/wiki/Renaissance_Technologies

Re: Numerai, a hedge fund built by a community of anonymous data scientists

#17

Post by joncooper »

"It is intuitively obvious that an open access hedge fund will generate more intelligence than a closed system built on a pre-internet, pre-cryptocurrency, pre-AI organizational design."

Really? Because the folks with the magic black box aren't capable of funding an Interactive Brokers account to keep 100% of their upside and 100% of their IP?

(Also: risk management and order handling are harder problems than signal generation.)

Re: Numerai, a hedge fund built by a community of anonymous data scientists

#18

Post by hkmurakami »

T-A wrote: post #16
dharmon wrote: post #15

Earlier quoted context omitted.

Even more than "a couple of years". About 15 years ago I worked at a day trading firm and we were writing models that used machine learning. At the time we thought of it more as "computational statistics", but its basically what is called ML now and taught in ML courses (although we didn't use Neural Nets). BTW, even in 2001 we were far from the first to do this.

Depending on where you draw the line between statistics and machine learning, it could be argued to have originated with Bachelier's thesis in 1900 [1], or Thorp's adaptation of Kelly's work first to gambling and then to finance in the early 60s (he may have been first to use computers for this kind of thing) [2] or maybe with James Simons' Renaissance Technologies in the early 80s [3]. [1] https://en.wikipedia.org/w…

Regarding Rentec, the only decent book on their history has been in "The Quants". Are there any others?

https://www.amazon.com/Quants-Whizzes-Conquered-Street-Destr...

Re: Numerai, a hedge fund built by a community of anonymous data scientists

#19

Post by rgbrgb »

So cool. This is the first time I've heard of homomorphic encryption. In my case, Open Listings has a lot of real estate data that we're not allowed to vend programmatically (sale prices, list prices, property characteristics). It would be interesting to be able to release this data in a legally encrypted form and let data scientists train predictors. We currently have an offer creation API that's being used by algorithmic investors but they have to get their data to decide what to bid on from another source. My immediate questions maybe someone here will know the answer to...

1) Is it legal to vend a dataset that is encrypted this way if you're not allowed to vend the original? The OP implies that it is, but that seems too good to be true.

2) Is there software purpose-built for this type of thing? What's good in this domain? Our stack is mostly ruby but we're polyglots.

Re: Numerai, a hedge fund built by a community of anonymous data scientists

#20

Post by ikeboy »

shostack wrote: post #11
davnn wrote: post #10

Interesting idea. Not that crowd driven investment algorithms are new, but I have not seen a machine learning one before. What really ennoys me about this kind of businesses is that they pay tiny prices and shut the competitions once they have found what they were looking for, however Numerai might be completely different in that regard and I wish them the best! Btw: The article kind of conveys the feeling as if mach…

Can you clarify on the "pay tiny prices and shut the competitions once they have found what they were looking for" comment? Has this happened before? Is there anything here that makes it seem like this wouldn't be the case here?

They only require you to give them predictions, not a model. So you can reuse the same model over and over if it works, and they won't be able to cut you out, if that's what you're asking.
Post reply on HN