Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Jane Street Market Prediction ($100k Kaggle competition)

kaggle.com

Post reply on HN
Print view View original

51–60 of 217 posts

Re: Jane Street Market Prediction ($100k Kaggle competition)

#51

Post by riazrizvi »

usmannk wrote: post #24

As a frequent Kaggler (perhaps too frequent... it's a bit addicting, in a way I'm sure others on HN will understand), I was fairly intrigued to see this one pop up in the competition list a few days ago. Finance shops have tried their hand at Kaggle before, but I think they've normally been out of their domain. e.g. Two Sigma recently did a reinforcement learning game competition. I'd caution the HN crowd not to expe…

Mathematical analysis of financial markets is more celebrated when applied to relative valuation of different assets, rather than prediction of the market. Black-scholes, for example, applied calculus with an underlying no-arbitrage assumption to create a thriving market in option pricing, by giving traders a mechanism to reduce risk and thereby reduce bid offer spreads. Same in fixed income, mortgage, and credit market assets over the years.

The problem with predicting absolute levels, is that there is a game theoretic aspect which undermines any mathematical trading strategy as soon as it is public. optimal game theory trading strategies don’t produce great results, and they are relatively trivial to identify. Instead strong profits in market long/short macro positions are mostly created by information advantages, which don’t really make for interesting Kaggle competitions. For example, big profits in macro trading have historically been consistently achieved by front running customer orders, by building timing advantages on top of trading infrastructure, by funding research analysts that inspect operations on the ground, by lobbying for regulations that change market directions and so on.

It’s very hard to tell if a best performing hedge funds that doesn’t have an unfair advantage, that declares its only using quantitative strategies, is in fact just a statistical anomaly with a hollow narrative.

Re: Jane Street Market Prediction ($100k Kaggle competition)

#52

Post by throwaway378692 »

renewiltord wrote: post #43
aaronblohowiak wrote: post #37

Earlier quoted context omitted.

I think you are significantly underestimating the cost per hour of jane street employees

I have first party information, two years out of date. Do you have contradictory first party information? Yes/no will be sufficient for me to adjust my priors and I will be grateful.

Package for just-graduated SWEs is about $400k.

Re: Jane Street Market Prediction ($100k Kaggle competition)

#53

Post by theptip »

renewiltord wrote: post #43
aaronblohowiak wrote: post #37

Earlier quoted context omitted.

I think you are significantly underestimating the cost per hour of jane street employees

I have first party information, two years out of date. Do you have contradictory first party information? Yes/no will be sufficient for me to adjust my priors and I will be grateful.

The correct metric is not what the employee’s salary is, but the opportunity cost of their time. If all your engineers are working on urgent stuff, and each engineer adds in $1-2m of revenue a year, then the cost to your business of taking them off feature work to do recruiting is not $150/hr.

Re: Jane Street Market Prediction ($100k Kaggle competition)

#54

Post by [deleted] »

throwaway378692 wrote: post #52
renewiltord wrote: post #43

Earlier quoted context omitted.

I have first party information, two years out of date. Do you have contradictory first party information? Yes/no will be sufficient for me to adjust my priors and I will be grateful.

Package for just-graduated SWEs is about $400k.

[deleted]

Re: Jane Street Market Prediction ($100k Kaggle competition)

#55

Post by rahimnathwani »

fractionalhare wrote: post #25
usmannk wrote: post #24

As a frequent Kaggler (perhaps too frequent... it's a bit addicting, in a way I'm sure others on HN will understand), I was fairly intrigued to see this one pop up in the competition list a few days ago. Finance shops have tried their hand at Kaggle before, but I think they've normally been out of their domain. e.g. Two Sigma recently did a reinforcement learning game competition. I'd caution the HN crowd not to expe…

Yes, it's overwhelmingly unlikely that the winning model will actually be a competitive trading strategy. Kaggle encourages a domain agnostic approach to modeling, in the sense that participants use sophisticated machine learning and statistical methods but typically have no domain expertise in the underlying data. This kind of approach to finance has historically performed poorly. [1] Good quantitative trading is us…

"have no domain expertise in the underlying data. This kind of approach to finance has historically performed poorly"

I recently read 'The man who solved the market', about Jim Simons and Renaissance Capital. The way the book tells it, looking for patterns without seeking domain expertise (e.g. ignoring fundamental valuation of equities) is exactly what Renaissance did, and it worked out very well.

Re: Jane Street Market Prediction ($100k Kaggle competition)

#56

Post by renewiltord »

throwaway378692 wrote: post #52
renewiltord wrote: post #43

Earlier quoted context omitted.

I have first party information, two years out of date. Do you have contradictory first party information? Yes/no will be sufficient for me to adjust my priors and I will be grateful.

Package for just-graduated SWEs is about $400k.

Thank you.

Re: Jane Street Market Prediction ($100k Kaggle competition)

#57

Post by renewiltord »

theptip wrote: post #53
renewiltord wrote: post #43

Earlier quoted context omitted.

I have first party information, two years out of date. Do you have contradictory first party information? Yes/no will be sufficient for me to adjust my priors and I will be grateful.

The correct metric is not what the employee’s salary is, but the opportunity cost of their time. If all your engineers are working on urgent stuff, and each engineer adds in $1-2m of revenue a year, then the cost to your business of taking them off feature work to do recruiting is not $150/hr.

Yes, of course. Reasoning for not using that is as follows: if conservative estimates yield a yes, you don't need to assume more.

I know salary (2 years out of date). I don't know oppo cost.

Re: Jane Street Market Prediction ($100k Kaggle competition)

#58

Post by fractionalhare »

rahimnathwani wrote: post #55
fractionalhare wrote: post #25

Earlier quoted context omitted.

Yes, it's overwhelmingly unlikely that the winning model will actually be a competitive trading strategy. Kaggle encourages a domain agnostic approach to modeling, in the sense that participants use sophisticated machine learning and statistical methods but typically have no domain expertise in the underlying data. This kind of approach to finance has historically performed poorly. [1] Good quantitative trading is us…

"have no domain expertise in the underlying data. This kind of approach to finance has historically performed poorly" I recently read 'The man who solved the market', about Jim Simons and Renaissance Capital. The way the book tells it, looking for patterns without seeking domain expertise (e.g. ignoring fundamental valuation of equities) is exactly what Renaissance did, and it worked out very well.

I can see why someone would characterize RenTech that way but it's not really fair to do so. There is a lot of mythos about how Simons hired computer scientists, mathematicians, signal processing and NLP experts, etc. When Mercer came over from IBM, he definitely contributed a significant amount of analytical expertise that was probably nonexistent in financial trading at the time (with the possible exception of the Ed Thorp diaspora). The astrophysicists RenTech hires every year bring new insights in ways to model and understand vast amounts of data with absurd dimensionality.

But all of this has to be utilized in the context of the data. The reality is that you're not going to develop a sophisticated options trading strategy without a strong understanding of what an option (and more generally, a derivative) is. You can't develop a viable statistical arbitrage strategy just by treating market microstructure as a blackbox signal to be solved with e.g. Fourier analysis. You can certainly find an edge in using fundamentally superior methods of analysis, but you still need to know what that data represents in the context of the market.

Don't be fooled: people working at firms like RenTech have a strong understanding of the underlying finance. It's just that they learned it on the job, because the ethos at these firms is that learning fundamental theory in math and statistics is harder than learning fundamental theory in finance. You don't have to take my word for it though. Read about one of the few strategies of RenTech's which has been publicized: https://www.bloomberg.com/opinion/articles/2014-07-22/senate.... Deutsche and RenTech didn't team up on this strategy (to fantastic success) by treating basket options as some kind of blackbox abstraction devoid of delta, gamma, theta and vega.

Re: Jane Street Market Prediction ($100k Kaggle competition)

#59

Post by paxys »

toomuchtodo wrote: post #2

Any model superior to what Jane Street is running is worth vastly more than the prizes they’re offering. If you prove such a model out, get licensed (SEC, FINRA) and start soliciting to manage assets. Disclaimer: Not investment advice. Not a lawyer, not your fiduciary.

Why do you think this competition will result in a model superior to what Jane Street is running?

Re: Jane Street Market Prediction ($100k Kaggle competition)

#60

Post by Tinyyy »

1helloworld1 wrote: post #38

Isn't it pretty well known in the finance world that using stale public information to predict the market is a fool's errand? Unless you have some kind of specialized non-public data (e.g satellite images of number of cars parked outside parking malls, number of cargo ships moving in and out), trying to predict the market with historical data does worse than "Just give me some monkeys, darts and a dart board".

That’s not necessarily true.
Post reply on HN