Live data from Hacker News

Jane Street Market Prediction ($100k Kaggle competition)

kaggle.com

111–120 of 217 posts

Re: Jane Street Market Prediction ($100k Kaggle competition)

#111
post #24

As a frequent Kaggler (perhaps too frequent... it's a bit addicting, in a way I'm sure others on HN will understand), I was fairly intrigued to see this one pop up in the competition list a few days ago. Finance shops have tried their hand at Kaggle before, but I think they've normally been out of their domain. e.g. Two Sigma recently did a reinforcement learning game competition. I'd caution the HN crowd not to expe…

We should also remember that Jane Street is primarily an ETF market maker. Their main business isn't betting on prices of stocks or managing a portfolio. I've only taken a quick look at the data, but the problem doesn't seem to be focused on their core competencies, but instead is much more general.

>I've only taken a quick look at the data, but the problem doesn't seem to be focused on their core competencies, but instead is much more general

How can you tell? All the features are completely anonymized.

Re: Jane Street Market Prediction ($100k Kaggle competition)

#112
post #62

Any model superior to what Jane Street is running is worth vastly more than the prizes they’re offering. If you prove such a model out, get licensed (SEC, FINRA) and start soliciting to manage assets. Disclaimer: Not investment advice. Not a lawyer, not your fiduciary.

You are ignoring execution, infrastructure and real-market conditions. The model is just one part of the game.

+1 so much this

Jane Street is a high frequency firm. In this domain execution & infrastructure matters as much if not more than your algo secret sauce.

What good is kagglers' favorites giant boosted bagged lightgbm/xgboost/neural ensembles which will take seconds to predict, when my FPGA or ASIC running in the same rack as exchange's matching engine can make a million trades in the meantime with a much simpler strategy involving nothing beyond freshman math (take a cue in XTX Markets' firm name, for example)

And in the longer timeframes, the edge is more often than not in the data than training algo.

Re: Jane Street Market Prediction ($100k Kaggle competition)

#113

Earlier quoted context omitted.

I've wanted to start learning about this for a while but I'm really not sure where to start. I have a degree in CS and Math so I'm not a total layman wrt the maths. Do you have any suggestions?

What's your goal?

To have fun exploring and learning.

Re: Jane Street Market Prediction ($100k Kaggle competition)

#114

Any model superior to what Jane Street is running is worth vastly more than the prizes they’re offering. If you prove such a model out, get licensed (SEC, FINRA) and start soliciting to manage assets. Disclaimer: Not investment advice. Not a lawyer, not your fiduciary.

Quantopian tried this. Didn't work out. Now they have been acquihired by Robin Hood.

Talk about the blind leading the blind...

Re: Jane Street Market Prediction ($100k Kaggle competition)

#115

If I build a model that actually works well, I'm using it to get rich, patent it, and sell it to the company for a lot more than $100k.

Maybe or maybe not. You may need to build up your trading infrastructure first, which entails among other things low-latency connectivity to different venues, negotiate good deals with brokers to get low trading fees etc. If it were that simple, all the quants would be working for themselves. Trading is not just about having good prediction. Also if you publish/share your algorithm, people will copy it and it will lose its edge.

If you are really good so that whereas others can only predict the future 0.1s but you can predict the future 5s with the same accuracy, then sure, you could trade from home over the Internet, and if you are much more accurate than others, especially when the market moves a lot, you don't need low fees to be competitive.

Re: Jane Street Market Prediction ($100k Kaggle competition)

#116
post #72
post #59

Earlier quoted context omitted.

Why do you think this competition will result in a model superior to what Jane Street is running?

The winner model will likely outperform anything that Jane Street could come up with this 130 feature set. With 3000+ competitors, the top 10 will likely be superior to what Jane Street can do in-house. Then an ensemble of the top solutions will be the best possible model anyone can come up with.

On a frozen feature set and evaluation dataset, sure. 3000+ competitors vs a few employees at JS who prepped this data. In light of very noisy data, the chances are not really on JS side here. But would the best model still be as useful on novel out of sample data is very questionable

Market prediction is low signal-to-noise problem, what's much more likely the contest will be an exercise in overfitting and the final shake up will be yuuuge.

Re: Jane Street Market Prediction ($100k Kaggle competition)

#117
post #86

I'm sorry, if you could build a model to predict markets, why will you post in to Kaggle to get $40k in prize instead of applying this model to your own broker account?

It is much harder to turn a model into a profitable trading strategy than people realize. Apart from transaction costs, risk management and market impact there are also a lot of small operational details which can make or break your execution. One example I vaguely recall was that the details of how a specific foreign exchange conducted its closing auction could make a substantial difference to a strategy that involv…

This is basically why I’ve never seriously considered doing it myself. I had a neat idea in about 2005 which I tested on historical data, and it beat buy-and-forget on every share I tried except for Google, and it only had one free parameter.

But, even if I’d implemented it perfectly, and even if the algorithm has survived the financial crash, it would’ve only worked if I could trade for free, and other people copying the algorithm would probably have made it stop working.

Re: Jane Street Market Prediction ($100k Kaggle competition)

#118
post #24

As a frequent Kaggler (perhaps too frequent... it's a bit addicting, in a way I'm sure others on HN will understand), I was fairly intrigued to see this one pop up in the competition list a few days ago. Finance shops have tried their hand at Kaggle before, but I think they've normally been out of their domain. e.g. Two Sigma recently did a reinforcement learning game competition. I'd caution the HN crowd not to expe…

Mathematical analysis of financial markets is more celebrated when applied to relative valuation of different assets, rather than prediction of the market. Black-scholes, for example, applied calculus with an underlying no-arbitrage assumption to create a thriving market in option pricing, by giving traders a mechanism to reduce risk and thereby reduce bid offer spreads. Same in fixed income, mortgage, and credit mar…

> relative valuation of different assets, rather than prediction of the market

But isn't prediction an inherent part of valuation?

Re: Jane Street Market Prediction ($100k Kaggle competition)

#119
post #104

I never heard of Jane street. I visited the website and saw a bunch of people crammed into an open office like chickens in a breeding factory. That is not attractive to me. Competitions like this just seem to me like a cheap way to outsource problem solving without having to pay anyone. Like bringing someone in for an interview, letting them solve your problem, and then telling them they are not a good fit while prof…

Knowing what/who Jane Street is massively changes perspective on this. They are definitely not a random shitty company that outsources their core business to some challenge. They are quite known if only for the fact they practically adopted OCaml, which is quite impressive considering their size. I highly recommend checking out some of their talks on youtube like this one: https://www.youtube.com/watch?v=gXdMFxGdako

Do they actually produce something valuable for society, or is it just the trading profit they are after?

Re: Jane Street Market Prediction ($100k Kaggle competition)

#120

Earlier quoted context omitted.

Mathematical analysis of financial markets is more celebrated when applied to relative valuation of different assets, rather than prediction of the market. Black-scholes, for example, applied calculus with an underlying no-arbitrage assumption to create a thriving market in option pricing, by giving traders a mechanism to reduce risk and thereby reduce bid offer spreads. Same in fixed income, mortgage, and credit mar…

> relative valuation of different assets, rather than prediction of the market But isn't prediction an inherent part of valuation?

>But isn't prediction an inherent part of valuation?

Doesn't need to be. You have wealth you have to store somehwere because it is not being spent now. Can store it as equities, debt (including a bank account), Paper cash money under the mattress, gold bars, BTC, etc etc.

Let's say you have the list of potential places to store it and can get a price/value ratio but with a common term of "X" in the value

  - A = 3.1/X
  - B = 2.8/X
  - C = 1.9/X
  - D = 4.2/X
Gotta put it somewhere if you aren't spending it. In the above you'd Choose "C". Lowest price to value ratio even though you don't know what the actual value is.

Yes, this is a massive simplification as it ignores diversification benefits and so on but there's the basics of the usefulness of valuation where you don't and can't know the value.

"prediction of the market" here means "Do you think the S&P500 is going up or down this year?" Warren Buffet completely ignores this, takes no view on it. Timing the market is very hard. Assuming the market will reflect the economy and the economy will grow because people work hard, are smart, we invent new things and population increases, then picking the best of what's available is the normal value investors route, a la Buffet as the most famous practioner.

Burton Malkiel's "A Random Walk Down Wall St." is still, IMHO the best summary of all the theory and practise of investing - excluding the quant funds like Renaissance. Maybe he has an update for them? Maybe it's also worth reading if it exists or maybe not. Don't know.

No idea if renaissance time the market or what.

edit: punctuation and the Malkiel recommendation.

Post reply on HN