Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Jane Street Market Prediction ($100k Kaggle competition)

kaggle.com

Post reply on HN
Print view View original

161–170 of 217 posts

Re: Jane Street Market Prediction ($100k Kaggle competition)

#161

Post by alexmingoia »

vatican_banker wrote: post #154
alexmingoia wrote: post #143

Late to the game. https://numer.ai is an entire hedge fund built around an anonymous ML prediction tournament. They solicit predictions, trade them, and reward the best performing ones. IIRC They’ve paid millions in prizes over the last few years. They also recently introduced Numerai Signals, where they pay for the performance of actual training data. So you can make money providing datasets that perform well.

Numerai’s intro video[1] makes pretty wild claims and seems to target a pretty specific group of people. Personally I’m put off by the language used in the video. Also > Stake on your model to earn cryptocurrency No thanks. I prefer dollars in my bank account. 1. https://youtu.be/GWeC2PK4yXQ

What claim do you take issue with?

Staking was introduced to reward actual market performance instead of performance on holdout data, and to prevent spam.

Cryptocurrency is the method of payment because it’s a global competition, and that’s the only way to pay everyone across countries. Originally rewards were in Bitcoin but switched to an ethereum token to facilitate staking.

Re: Jane Street Market Prediction ($100k Kaggle competition)

#162

Post by xapata »

logicslave wrote: post #147
xapata wrote: post #101

Earlier quoted context omitted.

That doesn't make sense unless there are very few viable small scale strategies, at which point they'd probably be difficult to identify. Your assertion might have been true before computers were able to help someone manage many strategies simultaneously.

No it makes a ton of sense. 100k is too small to have a researcher focus on full time. His compensation is probably 500k or more. Then add in fees, infra, cost of regulations, etc and small time strategies arent developed. Making money in the market is completely overrated from a difficulty perspective, the hard part is managing billions.

You're saying it'd require a salary of $500k or more to find someone competent enough to manage an investment opportunity worth $100k annually? That implies the investment opportunities are actually quite complicated. Obviously, opportunities exist for investment gains at all magnitudes. The question was whether they're easy enough for the average Jolanda. If Jolanda would require $500k annual salary to do the work, then, that easy opportunity doesn't really exist at all.

Alternately, if Jolanda doesn't need to make this opportunity her full-time job, then the investment company can pay her $500k to manage many of these small opportunities. After all, there must be many opportunities. Otherwise only 1 of the HN readers would be able to exploit it. In which case, again, it doesn't really exist at all.

Re: Jane Street Market Prediction ($100k Kaggle competition)

#164

Post by snicker7 »

In a trading competition, the best strategy is the riskiest strategy. This maximizes likelihood of making a lot of money ... and losing a lot of money. But in a competition, the left tail doesn't matter. Second place and last place are identical.

Ideally, a trading competition should penalize risky investments. But this is hard to do retrospectively, especially when evaluating algorithms.

Re: Jane Street Market Prediction ($100k Kaggle competition)

#165

Post by logicslave »

xapata wrote: post #162
logicslave wrote: post #147

Earlier quoted context omitted.

No it makes a ton of sense. 100k is too small to have a researcher focus on full time. His compensation is probably 500k or more. Then add in fees, infra, cost of regulations, etc and small time strategies arent developed. Making money in the market is completely overrated from a difficulty perspective, the hard part is managing billions.

You're saying it'd require a salary of $500k or more to find someone competent enough to manage an investment opportunity worth $100k annually? That implies the investment opportunities are actually quite complicated. Obviously, opportunities exist for investment gains at all magnitudes. The question was whether they're easy enough for the average Jolanda. If Jolanda would require $500k annual salary to do the work,…

Honestly, you seem like youre trying to reason this out with zero experience in the field. You cant just "run statistics to identify strategies". I'm not going to go into it further because you know nothing

Re: Jane Street Market Prediction ($100k Kaggle competition)

#166

Post by galaxyLogic »

ben_w wrote: post #117
tikhonj wrote: post #86

Earlier quoted context omitted.

It is much harder to turn a model into a profitable trading strategy than people realize. Apart from transaction costs, risk management and market impact there are also a lot of small operational details which can make or break your execution. One example I vaguely recall was that the details of how a specific foreign exchange conducted its closing auction could make a substantial difference to a strategy that involv…

This is basically why I’ve never seriously considered doing it myself. I had a neat idea in about 2005 which I tested on historical data, and it beat buy-and-forget on every share I tried except for Google, and it only had one free parameter. But, even if I’d implemented it perfectly, and even if the algorithm has survived the financial crash, it would’ve only worked if I could trade for free, and other people copyin…

Consider you have some extra money, what do you do with it? Do you put it (or keep it) in a bank-account? That is an investment choice you have. If you have more money than you need for living then you are already investing it in some way, maybe a bank-account. And you make this investment decision following some algorithm in your head. So if you algorithm was so great wouldn't it make sense to use it rather than put money into a low-interest bank-account?

My point is when it comes to investing, inaction IS action too. Therefore an investment algorithm does not need to beat the market. It just needs to beat doing nothing.

Consider that if you have earnings you can put money into an IRA account and then trade with it almost for free.

Re: Jane Street Market Prediction ($100k Kaggle competition)

#167

Post by eru »

riazrizvi wrote: post #51
usmannk wrote: post #24

As a frequent Kaggler (perhaps too frequent... it's a bit addicting, in a way I'm sure others on HN will understand), I was fairly intrigued to see this one pop up in the competition list a few days ago. Finance shops have tried their hand at Kaggle before, but I think they've normally been out of their domain. e.g. Two Sigma recently did a reinforcement learning game competition. I'd caution the HN crowd not to expe…

Mathematical analysis of financial markets is more celebrated when applied to relative valuation of different assets, rather than prediction of the market. Black-scholes, for example, applied calculus with an underlying no-arbitrage assumption to create a thriving market in option pricing, by giving traders a mechanism to reduce risk and thereby reduce bid offer spreads. Same in fixed income, mortgage, and credit mar…

Just as an aside: the kind of prediction you are describing as relatively boring is still really useful, because it improves markets as far as the rest of society is concerned.

But yeah, it might not be that useful for the funds themselves.

Re: Jane Street Market Prediction ($100k Kaggle competition)

#168

Post by eru »

riazrizvi wrote: post #76
georgeecollins wrote: post #63

Earlier quoted context omitted.

This! >> The problem with predicting absolute levels, is that there is a game theoretic aspect which undermines any mathematical trading strategy as soon as it is public. I took finance in Business school, coming from doing a lot of statistical analysis in a research lab. I hated my finance professors and there pseudo science. Pricing formulas work great until they don't. The problem is when they don't, they really d…

To clarify, the hedge fund LTCM in "When Genius Failed", collapsed not because it relied on arbitrage 'pricing formulae', rather because it failed to properly execute arbitrage trades. LTCM in being overly leveraged, relied on other market participants to maintain short term price alignment, which meant it was not arbitrage. Salomon's reduced its role as market-maker, maintaining short term price alignment, which inc…

'Why aren't they shouting' also has some things to say about LTCM. Broadly in the same vein. Those guys were leveraged to the gills and couldn't take market movements against them.

From what I've read elsewhere, if you had managed to hold an LTCM replica portfolio until expiry, you would have actually made money.

Re: Jane Street Market Prediction ($100k Kaggle competition)

#169

Post by eru »

blovescoffee wrote: post #70
riazrizvi wrote: post #51

Earlier quoted context omitted.

Mathematical analysis of financial markets is more celebrated when applied to relative valuation of different assets, rather than prediction of the market. Black-scholes, for example, applied calculus with an underlying no-arbitrage assumption to create a thriving market in option pricing, by giving traders a mechanism to reduce risk and thereby reduce bid offer spreads. Same in fixed income, mortgage, and credit mar…

I've wanted to start learning about this for a while but I'm really not sure where to start. I have a degree in CS and Math so I'm not a total layman wrt the maths. Do you have any suggestions?

I have a book called Empirical Asset Pricing by Wayne Ferson. Seems like as good a place to start as any.

If you want something online, check out https://www.quantopian.com/

Re: Jane Street Market Prediction ($100k Kaggle competition)

#170

Post by logicchains »

riazrizvi wrote: post #51
usmannk wrote: post #24

As a frequent Kaggler (perhaps too frequent... it's a bit addicting, in a way I'm sure others on HN will understand), I was fairly intrigued to see this one pop up in the competition list a few days ago. Finance shops have tried their hand at Kaggle before, but I think they've normally been out of their domain. e.g. Two Sigma recently did a reinforcement learning game competition. I'd caution the HN crowd not to expe…

Mathematical analysis of financial markets is more celebrated when applied to relative valuation of different assets, rather than prediction of the market. Black-scholes, for example, applied calculus with an underlying no-arbitrage assumption to create a thriving market in option pricing, by giving traders a mechanism to reduce risk and thereby reduce bid offer spreads. Same in fixed income, mortgage, and credit mar…

>The problem with predicting absolute levels, is that there is a game theoretic aspect which undermines any mathematical trading strategy as soon as it is public. optimal game theory trading strategies don’t produce great results, and they are relatively trivial to identify.

This isn't true if you're fast enough. Everybody knows how to do arbitrage but it's still extremely profitable if you're faster than everyone else. HFTs are consistently more profitable than slow trading firms, earning 40%+ returns, it's just a much more capacity-constrained form of trading so absolute returns are lower.

Post reply on HN