“Everyone (including LLMs) is a genius in a bull market.”
Apparently everyone (but Gemini).
Could Gemini end up being better over the longer term?
We gave 5 LLMs $100K to trade stocks for 8 months
101–110 of 319 posts
Re: We gave 5 LLMs $100K to trade stocks for 8 months
#102Trading in a nearly 20 year bull market and doing well is not an accomplishment.
Re: We gave 5 LLMs $100K to trade stocks for 8 months
#103There's also this thing going on right now: https://nof1.ai/leaderboard Results are... underwhelming. All the AIs are focused on daytrading Mag7 stocks; almost all have lost money with gusto.
Let me guess, the mystery model is theirs
It says "Undisclosed frontier AI Lab (not Nof1)"
Re: We gave 5 LLMs $100K to trade stocks for 8 months
#104Is it just prompting LLMs with "I have $100k to invest. Here are all publicly traded stocks and a few stats on them. Which stocks should I buy?" And repeat daily, rebalancing as needed?
This isn't the best use case for LLMs without a lot of prompt engineering and chaining prompts together, and that's probably more insightful than running them LLMs head-to-head.
Re: We gave 5 LLMs $100K to trade stocks for 8 months
#105They outperformed the S&P 500 but seem to be fairly well correlated with it. Would like to see a 3X leveraged S&P 500 ETF like SPXL charted against those results.
VTI gained over 10% in that time period so it wasn't much better.
Re: We gave 5 LLMs $100K to trade stocks for 8 months
#106The obvious next question is: does the AI on cocaine outperform? https://pihk.ai/
Re: We gave 5 LLMs $100K to trade stocks for 8 months
#107When I see stuff like this, I feel like rereading the Incerto by Taleb just to refresh and sharpen my bullshit senses.
Re: We gave 5 LLMs $100K to trade stocks for 8 months
#108I wouldn’t trust any backtracking test with these models. Try doing a real-time test over 8 months and see what happens then. I’d also be suspicious of anything that doesn’t take actual costs into account.
Re: We gave 5 LLMs $100K to trade stocks for 8 months
#109Is finding the right stocks to invest in an LLM problem? Language models aren't the right fit, I would presume. It would also be insightful to compare this with traditional ML models.
Re: We gave 5 LLMs $100K to trade stocks for 8 months
#110Earlier quoted context omitted.
> It would almost be more interesting to specifically train the model on half the available market data, then test it on another half. Yes, ideally you’d have a model trained only on data up to some date, say January 1, 2010, and then start running the agents in a simulation where you give them each day’s new data (news, stock prices, etc.) one day at a time.
I mean ultimately this is an exercise in frustration because if you do that you will have trained your model on market patterns that might not be in place anymore. For example after the 2008 recession regulations changed. So do market dynamics actually work the same in 2025 as in 2005? I honestly don’t know but intuitively I would say that it is possible that they do not. I think a potentially better way would be to…
> you will have trained your model on market patterns that might not be in place anymore
My working definition of technical analysis [0]