Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We gave 5 LLMs $100K to trade stocks for 8 months

aitradearena.com

Post reply on HN
Print view View original

141–150 of 319 posts

Re: We gave 5 LLMs $100K to trade stocks for 8 months

#141

Post by cap11235 »

seanmcdirmid wrote: post #135
hobobaggins wrote: post #130

Earlier quoted context omitted.

They didn't just outperform "normal" congress critters.. they also outperformed nearly every hedge fund on the planet. But they (meaning, of course, just one person and their spouse) are obviously geniuses.

Hedge funds suck though. They don’t invest in FAANG, they do risky stuff that doesn’t pay off, you are still comparing incomparable things. I’m obviously a genius because 90% of my stock is in tech, most of us on HN are geniuses in your opinion?

What do you think hedge funds do?

Re: We gave 5 LLMs $100K to trade stocks for 8 months

#142

Post by bmitc »

naet wrote: post #62

I used to work for a brokerage API geared at algorithmic traders and in my experience anecdotal experience many strategies seem to work well when back-tested on paper but for various reasons can end up flopping when actually executed in the real market. Even testing a strategy in real time paper trading can end up differently than testing on the actual market where other parties are also viewing your trades and makin…

I've honestly never understood what backtesting even does because of the things you mention like time it takes to request and close trades (if they even do!), responses to your trades, the continuous and dynamic input of the market into your model, etc.

Is there any reference that explains the deep technicalities of backtesting and how it is supposed to actually influence your model development? It seems to me that one could spend a huge amount of effort on backtesting that would distract from building out models and tooling and that that effort might not even pay off given that the backtesting environment is not the real market environment.

Re: We gave 5 LLMs $100K to trade stocks for 8 months

#143

Post by bmitc »

culi wrote: post #128
btbuildem wrote: post #121

It turns out DeepSeek only made BUY trades (not a single SELL in the history in their live example) -- so basically, buy & hold strategy wins, again.

this study should be replicated during a bear market

Buy and hold performs well over long time scales by simply not adjusting based upon sentiment.

Re: We gave 5 LLMs $100K to trade stocks for 8 months

#145

Post by lisbbb »

ipnon wrote: post #96
Nevermark wrote: post #60

Just one run per model? That isn't backtesting. I mean technically it is, but "testing" implies producing meaningful measures. Also just one time interval? Something as trivial as "buy AI" could do well in one interval, and given models are going to be pumped about AI, ... 100 independent runs on each model over 10 very different market behavior time intervals would producing meaningful results. Like actually credibl…

Yes, if these models available for $200/month a making 50% returns reliably, why isn’t Citadel having layoffs?

In my experience, you get a few big winners, but since you have to keep placing new trades (e.g. bets) you eventually blow one and lose most of what you made. This is particularly true with options and futures trades. It's a stupid way to speculate with or without AI help doesn't matter and will never matter.

Re: We gave 5 LLMs $100K to trade stocks for 8 months

#146

Post by lisbbb »

naet wrote: post #62

I used to work for a brokerage API geared at algorithmic traders and in my experience anecdotal experience many strategies seem to work well when back-tested on paper but for various reasons can end up flopping when actually executed in the real market. Even testing a strategy in real time paper trading can end up differently than testing on the actual market where other parties are also viewing your trades and makin…

This. This all day. I used to paper trade using ThinkOrSwim and I was doubling and tripling my money effortlessly. Then I decided to move my strategy to the real deal and it didn't do very well at all. It was all bs.

Re: We gave 5 LLMs $100K to trade stocks for 8 months

#147

Post by pottertheotter »

cheeseblubber wrote: post #39

OP here. We realized there are a ton of limitations with backtest and paper money but still wanted to do this experiment and share the results. By no means is this statistically significant on whether or not these models can beat the market in the long term. But wanted to give everyone a way to see how these models think about and interact with the financial markets.

Cool experiment.

I have a PhD in capital markets research. It would be even more informative to report abnormal returns (market/factor-adjusted) so we can tell whether the LLMs generated true alpha rather than just loading on tech during a strong market.

Re: We gave 5 LLMs $100K to trade stocks for 8 months

#148

Post by cyberrock »

olliepro wrote: post #19
bcrosby95 wrote: post #12

> Grok ended up performing the best while DeepSeek came close to second. Almost all the models had a tech-heavy portfolio which led them to do well. Gemini ended up in last place since it was the only one that had a large portfolio of non-tech stocks. I'm not an investor or researcher, but this triggers my spidey sense... it seems to imply they aren't measuring what they think they are.

A more sound approach would have been to do a monte carlo simulation where you have 100 portfolios of each model and look at average performance.

While not strictly stocks, it would be interesting to see them trade on game economies like EVE, WoW, RuneScape, Counter Strike, PoE, etc.

Re: We gave 5 LLMs $100K to trade stocks for 8 months

#149

Post by lisbbb »

lvspiff wrote: post #115

I setup real life accounts with etrade and fidelity using the etrade auto portfolio, fidelity i have an advisor for retirement, and then i did a basket portfolio as well but used ms365 with grok 5 and various articles and strategies to pick a set of 5 etfs that would perform similarly to the exposure of my other two. This year So far all are beating the s&p % wise (only by In other words though im not surprised at al…

How much are the expense ratios on those etfs you chose, though? I mean, Vanguard, Fidelity, Blackrock, and others have extremely low cost funds and etfs and it has been shown year after year and decade after decade that you can't beat their average returns over the long term. Indexing works for a reason. Beating something by 1%? It's not even worth it if your costs and taxes are higher than that.

Re: We gave 5 LLMs $100K to trade stocks for 8 months

#150

Post by fragmede »

scubbo wrote: post #134
UncleMeat wrote: post #123

Earlier quoted context omitted.

I know that Musk deserving a lifetime achievement award at the Adult Video Network awards over Riley Reid is definitely an indication of minimal "system prompt modification that distort[s] reality."

...I'm not familiar with the reference.

https://www.theguardian.com/technology/2025/nov/21/elon-musk...
Post reply on HN