Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Numerai – A hedge fund built by a global community of anonymous data scientists

numer.ai

Post reply on HN
Print view View original

81–90 of 120 posts

Re: Numerai – A hedge fund built by a global community of anonymous data scientists

#81

Post by Houshalter »

daveguy wrote: post #50
zitterbewegung wrote: post #3

Why should I think that being anonymous should give some advantage ? I think it would be more of a disadvantage due to should I trust you? Also, how do I know these people are truly anonymous ?

Anonymous in the sense that numerai does not ask for any personally identifying information. You could easily prove you are who you say you are in numerai. Also, I think the big benefit from a data scientist working on this is you can test methods with generic features, submit the results and get paid if they are good, but not submit any part of the methodology to any third party. If you can kill it on numerai then m…

But how do you trust the owners not to make fake accounts and only pay out to them?

Re: Numerai – A hedge fund built by a global community of anonymous data scientists

#82

Post by jamez1 »

Houshalter wrote: post #79
jamez1 wrote: post #68

Earlier quoted context omitted.

They could just be normalizing every data point to be between 0 and 1 by dividing by the range. That's a homomorphic encryption.. it passes your weird assumptions. I don't know why you're harping on about sound encryption, the point of this is to keep the statistical information intact in the cipher, without giving away the underlying market data.

It's very weird to call data normalization "encryption". This is just a standard procedure done on most datasets. Encryption implies they've gone to extra processing to make sure you can't figure out what the variables mean. I think it's either an abuse of the word 'encryption'. That, or they really have done something weird to this dataset. Which will probably make it useless for statistical algorithms. Even normali…

They only need to go as far as to obfuscate the market data this was derived from, so they don't have to pay exchange licencing fees.

It doesn't have to have an exponential time complexity on decryption to qualify as 'encryption'. Multiplying by 2 could be considered homomorphic encryption.

You might think encryption means something else and that it's an abuse of the word but unlike the spy novella that you derive this impression from, these guys actually are ex-spies.

Re: Numerai – A hedge fund built by a global community of anonymous data scientists

#83

Post by shoegumfoot »

Houshalter wrote: post #77
wyager wrote: post #70

Earlier quoted context omitted.

> than this data juggling deep learning bullshit which results in profits being paid out to a bunch of intelligent, greedy and unwise people. Holy anti-intellectualism, Batman! If you describe machine learning as "data juggling bullshit", this is very strong evidence that you simply don't understand what it is. This is an indictment of you, not of machine learning. Machine learning would more accurately be called "ap…

I read it mainly as a criticism of this site and this application of deep learning. Predicting stock prices slightly better doesn't really make the world a better place or improve anyone's lives. But it does consume the time and resources of very intelligent people who might be able to do incredible things elsewhere. Particularly the problem of this website, is it's huge focus on anonymity. You don't need to expose y…

Don't you think that is says something (not terribly good) about society that these very intelligent people must spend their time this way if they want to be rewarded?

Why spend 80 hours a week chasing grant money when you can spend 60 moving numbers around?

Re: Numerai – A hedge fund built by a global community of anonymous data scientists

#84

Post by jamez1 »

mortehu wrote: post #73
jamez1 wrote: post #67

Earlier quoted context omitted.

In his case, these were 5 funds of hedge funds. Which underneath could be much more than 20 funds. You're also forgetting how much the compounding of the fees can cost, and you have two layers of them. This could easily make the S&P at least 3x higher in returns after 10 years.

Aha, I had skimmed past the fund of fund parts. In CFA Institute's estimate, the difference in fees only made up about half of the underperformance at the end of 2014, especially considering that 2008 was an easy win for hedge funds: https://blogs.cfainstitute.org/investor/2015/02/12/betting-w... Note that at the end of 2015, eight years had elapsed, not ten.

Fair enough, there is a slight bit more to the buffet bet than I trivialized.

The short portion of the hedge fund will have been losing as we are in a bull market. The full length of a cycle should see this effect reverse (and indeed 2008 shows this).

So Buffet did have to pick a bull market in making this bet, but he still had better than 50/50 odds regardless.

You also have other costs specific to hedge funds over active funds, such as higher brokerage from more frequent trades, and short interest. These have the same drag as the management fees, and I should have included that also in the description.

Re: Numerai – A hedge fund built by a global community of anonymous data scientists

#85

Post by botexpert »

solaarphunk wrote: post #61
lordnacho wrote: post #16

Hedge fund guy here. - You need to know something about the domain in order to make sensible predictions. Is the data daily? Is it per second? Is it ticks? You can't build a sensible model if you don't know that, even if you have good predictions. Relative cost will vary a lot between timescales. - It matters what the features are. Maybe there's some clever reason why it doesn't, but until I hear why I'm going to tak…

Although numerai could control for some of the factors that you mention above, I'm pretty sure they are falling into the data-mining fallacy that most people do when they approach quantitative trading from the outside world. I completely agree with your skepticism of their approach. While prediction based on data may be valuable in some cases, it isn't robust enough to scale up in any meaningful way. Context matters,…

Knowing what features represent is an advantage but isn't necessary. At least that's what nn research shows. But currently a lot of data of required for good representation learning. Ensembling also works, or can work, and it seems that's what they're doing. Although it doesn't look like datasets are large enough for NNs out that ensembles are large enough for good prediction.

Re: Numerai – A hedge fund built by a global community of anonymous data scientists

#86

Post by mr_pink »

neffy wrote: post #74
delegate wrote: post #26

"It's just a pure math problem. It's like a math competition. You don't need to know anything about finance, you don't know anything about hedge funds... you don't even have to speak English..." This is not a pure math problem. Eventually the outcome of all these models and predictions affects the stock prices and - if it becomes as successful as you hope - the economy as a whole. And the physical world: people, anim…

Agreed - it is a terrible waste of talent. Ironically, it's also not entirely a math problem - the underlying system doesn't exactly follow the expected rules of math.

What expected "rules of math" does it not follow?

Re: Numerai – A hedge fund built by a global community of anonymous data scientists

#87

Post by delegate »

wyager wrote: post #70
delegate wrote: post #26

"It's just a pure math problem. It's like a math competition. You don't need to know anything about finance, you don't know anything about hedge funds... you don't even have to speak English..." This is not a pure math problem. Eventually the outcome of all these models and predictions affects the stock prices and - if it becomes as successful as you hope - the economy as a whole. And the physical world: people, anim…

> than this data juggling deep learning bullshit which results in profits being paid out to a bunch of intelligent, greedy and unwise people. Holy anti-intellectualism, Batman! If you describe machine learning as "data juggling bullshit", this is very strong evidence that you simply don't understand what it is. This is an indictment of you, not of machine learning. Machine learning would more accurately be called "ap…

> What makes you think that the people using applied statistics to make money are "unwise"?

Because it is an incredible waste of talent.

I know this is in contradiction with all the 'values' that have been drilled into our minds since we were born, but it's about time we wake up and reorder our priorities.

>Based on your tone, I would guess that it's because what they're doing doesn't agree with your folk definition of "an honest day's work" or something like that.

If you look at the state of the biosphere / atmosphere / oceans - the data - and if you have children - then it should be quite obvious.

Also there are the socio-economic challenges that the world faces right now - in fact it is unclear if we're going to make it to the next century as a species.

It really don't matter how much "money" you have in your account when your city sinks under the ocean...

Re: Numerai – A hedge fund built by a global community of anonymous data scientists

#88

Post by hueving »

delegate wrote: post #87
wyager wrote: post #70

Earlier quoted context omitted.

> than this data juggling deep learning bullshit which results in profits being paid out to a bunch of intelligent, greedy and unwise people. Holy anti-intellectualism, Batman! If you describe machine learning as "data juggling bullshit", this is very strong evidence that you simply don't understand what it is. This is an indictment of you, not of machine learning. Machine learning would more accurately be called "ap…

> What makes you think that the people using applied statistics to make money are "unwise"? Because it is an incredible waste of talent. I know this is in contradiction with all the 'values' that have been drilled into our minds since we were born, but it's about time we wake up and reorder our priorities. >Based on your tone, I would guess that it's because what they're doing doesn't agree with your folk definition…

>It really don't matter how much "money" you have in your account when your city sinks under the ocean..

You just move. That's what money enables. No city is going to sink so rapidly people just drown.

Re: Numerai – A hedge fund built by a global community of anonymous data scientists

#89

Post by inimino »

hueving wrote: post #88
delegate wrote: post #87

Earlier quoted context omitted.

> What makes you think that the people using applied statistics to make money are "unwise"? Because it is an incredible waste of talent. I know this is in contradiction with all the 'values' that have been drilled into our minds since we were born, but it's about time we wake up and reorder our priorities. >Based on your tone, I would guess that it's because what they're doing doesn't agree with your folk definition…

>It really don't matter how much "money" you have in your account when your city sinks under the ocean.. You just move. That's what money enables. No city is going to sink so rapidly people just drown.

So you're literally arguing that we just need to focus on making money, and ignore things like the environment? Yep, sounds about like a textbook example of "unwise" to me.

Re: Numerai – A hedge fund built by a global community of anonymous data scientists

#90

Post by xg15 »

hueving wrote: post #88
delegate wrote: post #87

Earlier quoted context omitted.

> What makes you think that the people using applied statistics to make money are "unwise"? Because it is an incredible waste of talent. I know this is in contradiction with all the 'values' that have been drilled into our minds since we were born, but it's about time we wake up and reorder our priorities. >Based on your tone, I would guess that it's because what they're doing doesn't agree with your folk definition…

>It really don't matter how much "money" you have in your account when your city sinks under the ocean.. You just move. That's what money enables. No city is going to sink so rapidly people just drown.

So the 2012 (the movie) way of solving environmental problems: The cities can drown as long as we don't belong to the people who drown with it.
Post reply on HN