Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We gave 5 LLMs $100K to trade stocks for 8 months

aitradearena.com

Post reply on HN
Print view View original

151–160 of 319 posts

Re: We gave 5 LLMs $100K to trade stocks for 8 months

#151

Post by seanmcdirmid »

cap11235 wrote: post #141
seanmcdirmid wrote: post #135

Earlier quoted context omitted.

Hedge funds suck though. They don’t invest in FAANG, they do risky stuff that doesn’t pay off, you are still comparing incomparable things. I’m obviously a genius because 90% of my stock is in tech, most of us on HN are geniuses in your opinion?

What do you think hedge funds do?

They use crazy investment strategies that allow them to capture high returns in adverse general market conditions, but they rather under perform the general market in normal and booming conditions. “Hedge” is actually in their name for a reason. Rich people use hedge funds for…hedging.

Re: We gave 5 LLMs $100K to trade stocks for 8 months

#153

Post by stouset »

chris_st wrote: post #110
IgorPartola wrote: post #98

Earlier quoted context omitted.

I mean ultimately this is an exercise in frustration because if you do that you will have trained your model on market patterns that might not be in place anymore. For example after the 2008 recession regulations changed. So do market dynamics actually work the same in 2025 as in 2005? I honestly don’t know but intuitively I would say that it is possible that they do not. I think a potentially better way would be to…

> you will have trained your model on market patterns that might not be in place anymore My working definition of technical analysis [0] [0]: https://en.wikipedia.org/wiki/Technical_analysis

I am frankly astonished at the number of otherwise-intelligent people who actually seem to believe in this stuff.

One of the worst possible things to do in a competitive market is to trade by some publicly-available formulaic strategy. It’s like announcing your rock-paper-scissors move to your opponent in advance.

Re: We gave 5 LLMs $100K to trade stocks for 8 months

#154

Post by stouset »

hobobaggins wrote: post #130
seanmcdirmid wrote: post #111

Earlier quoted context omitted.

We had this discussion in previous posts about congressional leaders who had the risk appetite to go tech heavy and therefore outperformed normal congress critters. Going heavy on tech can be rewarding, but you are taking on more risk of losing big in a tech crash. We all know that, and if you don't have that money to play riskier moves, its not really a move you can take. Long term it is less of a win if a tech bubb…

They didn't just outperform "normal" congress critters.. they also outperformed nearly every hedge fund on the planet. But they (meaning, of course, just one person and their spouse) are obviously geniuses.

Hedge funds’ goals are often not to maximize profit, but to provide returns uncorrelated with the rest of some benchmark market. This is useful for the wealthy as it means you can better survive market crashes.

Re: We gave 5 LLMs $100K to trade stocks for 8 months

#156

Post by apparent »

apparent wrote: post #66
cheeseblubber wrote: post #39

OP here. We realized there are a ton of limitations with backtest and paper money but still wanted to do this experiment and share the results. By no means is this statistically significant on whether or not these models can beat the market in the long term. But wanted to give everyone a way to see how these models think about and interact with the financial markets.

> Grok ended up performing the best while DeepSeek came close to second. I think you mean "DeepSeek came in a close second".

OK, now it says:

> Grok ended up performing the best while DeepSeek came close second.

"came in a close second" is an idiom that only makes sense word-for-word.

Re: We gave 5 LLMs $100K to trade stocks for 8 months

#157

Post by taneq »

IgorPartola wrote: post #117
chris_st wrote: post #110

Earlier quoted context omitted.

> you will have trained your model on market patterns that might not be in place anymore My working definition of technical analysis [0] [0]: https://en.wikipedia.org/wiki/Technical_analysis

It is always fun (in a broad sense of that word) when I make a comment on an industry I know nothing about and somehow stumble onto a thing that not only has a name but also research. I am sure there is a German word for that feel of discovering something that countless others have already discovered.

Any time I invent a cool thing, I go and try and find it online. Usually it's already an established product, which totally validates my feeling that the thing I invented is cool and would be a good product. :D

Occasionally it's (as far as I can tell) a legitimately new 'wow that's obvious' style thing and I consider prototyping it. :)

Re: We gave 5 LLMs $100K to trade stocks for 8 months

#159

Post by Marsymars »

Nevermark wrote: post #60

Just one run per model? That isn't backtesting. I mean technically it is, but "testing" implies producing meaningful measures. Also just one time interval? Something as trivial as "buy AI" could do well in one interval, and given models are going to be pumped about AI, ... 100 independent runs on each model over 10 very different market behavior time intervals would producing meaningful results. Like actually credibl…

To take it to the absurdist conclusion, you could backtest each LLM "which single stock should I buy on Jan 1, 2010 to maximize my returns over the next 15 years?"

If your backtested LLM performed well, would you use the same strategy for the next 15 years? (I suppose there are people who would.)

Re: We gave 5 LLMs $100K to trade stocks for 8 months

#160

Post by Marsymars »

halzm wrote: post #23

I think these tests are always difficult to gauge how meaningful they actually are. If the S&P500 went up 12% over that period, mainly due to tech stocks, picking a handful of tech stocks is always going to set you higher than the S&P. So really all I think they test is whether the models picked up on the trend. I more surprised that Gemini managed to lose 10%. I wish they actually mentioned what the models invested…

> picking a handful of tech stocks is always going to set you higher than the S&P.

That's a bold claim.

Post reply on HN