Hyperliquid now has select tokenized equities as well. Would love to see how these models perform when trading equities
I've been following these for a while and many of the trades taken by DeepSeek and Qwen were really solid
Exploring the limits of large language models as quant traders
51–60 of 103 posts
Re: Exploring the limits of large language models as quant traders
#52. . . "The (geometric) average result at the end seems to be that the LLMs are down 35 % from their initial capital – and they got there in just 96 model-days. That's a daily return of -0.6 %, or a yearly return of -81 %, i.e. practically wiping out the starting capital." Proves that LLM's are nowhere near close to AGI.
The vast majority of intelligent humans cannot profitably trade on intraday
timeframes
Re: Exploring the limits of large language models as quant traders
#53Super interesting! You can click the "live" link in the header to see how they performed over time. The (geometric) average result at the end seems to be that the LLMs are down 35 % from their initial capital – and they got there in just 96 model-days. That's a daily return of -0.6 %, or a yearly return of -81 %, i.e. practically wiping out the starting capital. Although I lack the maths to determine it numerically (…
Agreed, and I'd also love to see a baseline of human performance here, both of experienced quant traders and of fresh grads who know the theory but never did this sort of trading and aren't familiar with the crypto futures market.
As someone who trades crypto semi-professionally, this was one of the toughest trading periods I've ever seen and included a massive liquidation event on 10th of October that wiped out over $20B in capital. Any trader who broke even in this period likely outperformed. I know some very, very good traders who got wiped out on leverage on 10th of October when stop losses didn't trigger and prices plummetted to 2021 levels (still no clarity why).
BTC also performed abysmally during this period with a sustained chop down from $126k to $90k.
Re: Exploring the limits of large language models as quant traders
#54At the end of the day it all comes down to input data. There are a lot of things you can do to collect proprietary data to give you an edge.
That's funny because that advice is _directly_ counter to what most HFT quants say
Re: Exploring the limits of large language models as quant traders
#55Today it's clear that there are limitations to LLM's. But I also see this incredible growth curve to LLM's improvement. 2 years ago, I wouldn't expect llm's to one shot a web application or help me debug obscure bugs and 2 years later I've been proven wrong. I completely believe that trading is going to be saturated with ai traders in the future. And being able to predict and detect ai trading patterns is going to be…
> I completely believe that trading is going to be saturated with ai traders in the future
That's probably good news for us index fund investors. We need people to believe they're going to beat the market.
Re: Exploring the limits of large language models as quant traders
#56At the end of the day it all comes down to input data. There are a lot of things you can do to collect proprietary data to give you an edge.
That's funny because that advice is _directly_ counter to what most HFT quants say
Right, because they will tell you exactly how they generate alpha for all the world to see. It’s worth mentioning quant is not all HFT.
Re: Exploring the limits of large language models as quant traders
#57I was chatting to a friend in the space. This guy is both experienced in trading and LLMs, and has gone all-in on using LLMs to get his day-to-day coding done. Now he's working on the model to end all models, which is a fairly ambitious way to put it, but it throws off some interesting conversations. You need domain knowledge to get this to work. Things like "we fed the model the market data" are actually non-obvious…
You can vibe code in this space as an individual because practically everything you are going to write is already in the training data.
The big Quant hedge funds have been using machine learning for decades. I took the coursera RL in finance class years ago.
The idea you are going to beat Two Sigma at their own game with tokens is just an absurdity.
Personally, I think any individual on their own that claims they are doing anything in the algorithmic / ML high frequency space is full of shit.
I could talk like I am too and sound really impressive to someone outside the space. That is much different though than actually making money on what you claim you are doing.
It reminds me of an artist friend when I was younger. She was an artist and I quite liked her paintings. She would tell everyone she is an artist. She was also an encyclopedia when it came to anything art related. She wasn't actually selling much art though. She lived off the $10k a month allowance her rich father gave her. She wasn't even being dishonest but when you didn't know the full picture a person would just assume she was living off her art sales.
Re: Exploring the limits of large language models as quant traders
#58Earlier quoted context omitted.
they're tools. treat them as tools. since they're so general, you need to explore if and how you can use them in your domain. guessing 'they're poorly suited' is just that, guessing. in particular: > We also found that the models were highly sensitive to seemingly trivial prompt changes this is as much as obvious for anyone who seriously looked at deploying these, that's why there are some very successful startups in…
> guessing 'they're poorly suited' is just that, guessing I have a really nice bridge to sell you... This "failure" is just a grab at trying to look "cool" and "innovative" I'd bet. Anyone with a modicum of understanding of the tooling (or hell experience they've been around for a few years now, enough for people to build a feeling for this), knows that this it's not a task for a pre-trained general LLM.
I think you have a different idea of what I'm saying than what I'm actually saying.
Re: Exploring the limits of large language models as quant traders
#59Earlier quoted context omitted.
Agreed, and I'd also love to see a baseline of human performance here, both of experienced quant traders and of fresh grads who know the theory but never did this sort of trading and aren't familiar with the crypto futures market.
As someone who trades crypto semi-professionally, this was one of the toughest trading periods I've ever seen and included a massive liquidation event on 10th of October that wiped out over $20B in capital. Any trader who broke even in this period likely outperformed. I know some very, very good traders who got wiped out on leverage on 10th of October when stop losses didn't trigger and prices plummetted to 2021 leve…
Note that 10th of October is before the trading period in this experiment. If anything, autoregression over shorter timescales would suggest entering after 10th of October being a good idea!
Re: Exploring the limits of large language models as quant traders
#60Earlier quoted context omitted.
The Asirra paper isn't from a ML research group. The statement: "Barring a major advance in machine vision, we expect computers will have no better than a 1/54,000 chance of solving it" is just a statement of fact - it wasn't any forms of prediction. If you read the paper you note that they surveyed researchers about the current state of the art ("Based on a survey of machine vision literature and vision ex- perts at…
Maybe I misunderstand, but it seems that there's nothing in your comment that contradicts any aspect of mine. Regarding image classification. As I see it, a company like Microsoft surveying researchers about the state of the art and then making a business call to recommend the use of it as a captcha is significantly more meaningful of a prediction than any single paper from an ML research group. My intent was just to…
> making a business call to recommend the use of it as a captcha is significantly more meaningful of a prediction than any single paper from an ML research group.
That's not what this is. It's a research paper from 3 researchers at MSR.