Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We gave 5 LLMs $100K to trade stocks for 8 months

aitradearena.com

Post reply on HN
Print view View original

291–300 of 319 posts

Re: We gave 5 LLMs $100K to trade stocks for 8 months

#291

Post by Maxatar »

chroma205 wrote: post #284
Maxatar wrote: post #257

Earlier quoted context omitted.

>Your order can legally be “front run” by the lead or designated market maker who receives priority trade matching, bypassing the normal FIFO queue. Not all exchanges do this. Unless you're thinking of some obscure exchange in a tiny market, this is just untrue in the U.S., Europe, Canada, and APAC. There are no exchanges where market makers get any kind of priority to bypass the FIFO queue.

> There are no exchanges where market makers get any kind of priority to bypass the FIFO queue. Nope, several large, active, and liquid markets in the US. Legally it’s not named “bypass the FIFO queue”. That would be dumb. In practice, it goes by politically correct names such as “designated market maker fill” or “institutional order prioritization” or “leveling round”.

I can tell you as someone who is a designated market maker on several ETFs in the U.S., none of this exists as a means of giving market makers priority fills. You're taking existing terms and misusing them. For example institutional order prioritization is used as a wash trade prevention mechanism, not as a way for designated market makers to get some kind of fill preference. Leveling rounds also do not involve exchanges, this is an internal tool used by a broker's OMS to rebalance residuals so accounts end up with the intended allocation, or cleaning up odd-lot/mixed-lot leftovers.

I am getting the feeling you either are not actually a quant, or you were a quant and just misheard and confused a lot of things together, but one thing is for sure... your claim that market makers get some kind of priority fills is factually incorrect.

Re: We gave 5 LLMs $100K to trade stocks for 8 months

#292

Post by bmitc »

throwawayffffas wrote: post #223
bmitc wrote: post #143

Earlier quoted context omitted.

Buy and hold performs well over long time scales by simply not adjusting based upon sentiment.

Operating word is long, historically if you entered the market just before a downturn, it could take years up to a couple of decades to make up. Depending on which downturn we are looking at.

I think that requires entering once. I was referring to continuing to enter periodically and holding.

Re: We gave 5 LLMs $100K to trade stocks for 8 months

#293

Post by bmitc »

Maxatar wrote: post #258
bmitc wrote: post #142

Earlier quoted context omitted.

I've honestly never understood what backtesting even does because of the things you mention like time it takes to request and close trades (if they even do!), responses to your trades, the continuous and dynamic input of the market into your model, etc. Is there any reference that explains the deep technicalities of backtesting and how it is supposed to actually influence your model development? It seems to me that o…

We use back testing at my firm for two primary reasons, one as a way to verify correctness and two as a way to assess risk. We do not use it as a way to determine profitability.

This is interesting because I'm not immediately sure how you verify correctness and assess risk without also addressing profitability.

By assessing risk is that just checking that it does dump all your money and that you can at least maintain a stable investment cache?

Are you willing to say more about correctness? Is the correctness of the models, of the software, or something else?

Re: We gave 5 LLMs $100K to trade stocks for 8 months

#294

Post by snapdeficit »

Anyone who traded tech stocks in the 1990s when AmeriTrade appeared remembers this story.

Have the LLMS trade anything BUT tech stocks and see how they do.

That’s the real test.

EDIT: I remember this is probably before AmeriTrade offered options. I was calling in trades at 6:30AM PST to my broker while he probably laughed at me. But the point is the same: any doofus could make money buying tech stocks and holding for a few weeks. Companies were splitting constantly.

Re: We gave 5 LLMs $100K to trade stocks for 8 months

#295

Post by Maxatar »

bmitc wrote: post #293
Maxatar wrote: post #258

Earlier quoted context omitted.

We use back testing at my firm for two primary reasons, one as a way to verify correctness and two as a way to assess risk. We do not use it as a way to determine profitability.

This is interesting because I'm not immediately sure how you verify correctness and assess risk without also addressing profitability. By assessing risk is that just checking that it does dump all your money and that you can at least maintain a stable investment cache? Are you willing to say more about correctness? Is the correctness of the models, of the software, or something else?

Profitability is not in any way considered a property of the correctness of an algorithm. An algorithm can be profitable and incorrect, and an algorithm can be correct but not profitable.

Correctness has to do with whether the algorithm performed the intended actions in response to the inputs/events provided to it, nothing more. For the most part correctness of an algorithm can be tested the same way most software is tested, ie. unit tests, but it's also worth testing the algorithm using live data/back testing it since it's not feasible to cover every possible scenario in giant unit tests, but you can get pretty good coverage of a variety of real world scenarios by back testing.

Re: We gave 5 LLMs $100K to trade stocks for 8 months

#296

Post by [deleted] »

chroma205 wrote: post #164
naet wrote: post #62

I used to work for a brokerage API geared at algorithmic traders and in my experience anecdotal experience many strategies seem to work well when back-tested on paper but for various reasons can end up flopping when actually executed in the real market. Even testing a strategy in real time paper trading can end up differently than testing on the actual market where other parties are also viewing your trades and makin…

>but for various reasons can end up flopping when actually executed in the real market. 1. Your order can legally be “front run” by the lead or designated market maker who receives priority trade matching, bypassing the normal FIFO queue. Not all exchanges do this. 2. Market impact. Other participants will cancel their order, or increase their order size, based on your new order. And yes, the algos do care about your…

[deleted]

Re: We gave 5 LLMs $100K to trade stocks for 8 months

#297

Post by lisbbb »

1a527dd5 wrote: post #46

Time. That has been the best way to get returns. I setup a 212 account when I was looking to buy our first house. I bought in small tiny chunks of industry where I was comfortable and knowledgeable in. Over the years I worked up a nice portfolio. Anyway, long story short. I forgot about the account, we moved in, got a dog, had children. And then I logged in for the first time in ages, and to my shock. My returns were…

Yeah, uh, all I did was buy BRK.B like a decade ago and it's up 172% or something like that.

The only way I have seen people outperform is by having insider information.

Re: We gave 5 LLMs $100K to trade stocks for 8 months

#298

Post by buu700 »

skeeter2020 wrote: post #226
buu700 wrote: post #179

Earlier quoted context omitted.

Anyone who hasn't used Grok might be surprised to learn that it isn't shy about disagreeing with Elon on plenty of topics, political or otherwise. Any insinuation to the contrary seems to be pure marketing spin on his part. Grok is often absurdly competent compared to other SOTA models, definitely not a tool I'd write off over its supposed political leanings. IME it's routinely able to solve problems where other mode…

it's so wildly inconsistent you can't build on top of it with reliability. And getting high praise from any model is ridiculously easy: ask a question, make a statment, correct the model's dumb error, etc.

It's easy for us as humans to correct dumb mistakes made by AI. It's less easy for AI to correct mistakes made by AI.

What's remarkable on Grok's part is when it spends five minutes churning through a few thousand lines of code (not the whole codebase, just the relevant files) and correctly arrives at the correct root cause of a complex bug in one shot.

Grok as a model may or may not be uniquely amazing per se, but the service's eagerness to throw compute at problems that genuinely demand it is a superpower that makes at least makes it uniquely amazing in practice. By comparison, even Gemini 3 often returns lazy/shallow/wrong responses (and I say that as a regular user of Gemini).

Re: We gave 5 LLMs $100K to trade stocks for 8 months

#299

Post by brendoelfrendo »

solotronics wrote: post #264
hxtk wrote: post #175

Earlier quoted context omitted.

I suspect trading firms have already done this to the maximum extent that it's profitable to do so. I think if you were to integrate LLMs into a trading algorithm, you would need to incorporate more than just signals from the market itself. For example, I hazard a guess you could outperform a model that operates purely on market data with a model that also includes a vector embedding of a selection of key social and…

The part people are missing here is that if the trading firms are all doing something, that in itself influences the market. If they are all giving the LLMs money to invest and the AIs generally buy the same group of stocks, those stocks will go up. As more people attempt the strategy it infuses fresh capital and more importantly signaling to the trading firms there are inflows to these stocks. I think its probably a…

They could have the AI perform paper trading: give it a simulated account but real data. This would make sense to me if it was just a research project. That said, I imagine the more high-tech trading firms started running this research a long time ago and wouldn't be surprised if there were already LLM-based trading bots that could be influencing the market.

Re: We gave 5 LLMs $100K to trade stocks for 8 months

#300

Post by dhosek »

philipwhiuk wrote: post #248
rallies wrote: post #178

Earlier quoted context omitted.

We're running some live experiments these days, for both stocks and options. https://rallies.ai/arena

With actual money? Or still fake money?

Fake money is better than nothing, but one hopes that at the very least they’re correctly managing prices with the bid-ask spread, although real money would tend to influence what the actual numbers would be (small dollar amounts likely getting worse pricing, large dollar amounts potentially impacting the movement of the market).
Post reply on HN