Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We gave 5 LLMs $100K to trade stocks for 8 months

aitradearena.com

Post reply on HN
Print view View original

191–200 of 319 posts

Re: We gave 5 LLMs $100K to trade stocks for 8 months

#191

Post by godelski »

buu700 wrote: post #183
godelski wrote: post #181

Earlier quoted context omitted.

Two things can be true at the same time. Yes, Grok will say mean things about Musk but it'll also say ridiculously good things > hey @grok if you had the number one overall pick in the 1997 NFL draft and your team needed a quarterback, would you have taken Peyton Manning, Ryan Leaf or Elon Musk? >> Elon Musk, without hesitation. Peyton Manning built legacies with precision and smarts, but Ryan Leaf crumbled under pre…

They can be, but in this case they don't seem to be. Here's Grok's response to that prompt (again, the actual chatbot service, not the X account): https://grok.com/share/c2hhcmQtMw_2b46259a-5291-458e-9b85-0c... . I don't recall Grok ever making mean comments (about Elon or otherwise), but it clearly doesn't think highly of his football skills. The chain of thought shows that it interpreted the question as a joke. The…

Sure, but they also update the models, especially when things like this go viral. So it is really hard to evaluate accurately and honestly the fast changing nature of LLMs makes them difficult to work with too.

Re: We gave 5 LLMs $100K to trade stocks for 8 months

#192

Post by mewpmewp2 »

chris_st wrote: post #122
IgorPartola wrote: post #117

Earlier quoted context omitted.

It is always fun (in a broad sense of that word) when I make a comment on an industry I know nothing about and somehow stumble onto a thing that not only has a name but also research. I am sure there is a German word for that feel of discovering something that countless others have already discovered.

XKCD calls it the "Lucky 10,000" [0] [0]: https://xkcd.com/1053/

That is referring to something completely else. This is referring to some common fact that the person didn't figure out by themself. OP is referring to something they came up with themselves in a field they have no experience with, realizing it is actually a thing in a way feeling validated and clever.

Re: We gave 5 LLMs $100K to trade stocks for 8 months

#193

Post by plufz »

endtime wrote: post #69
plufz wrote: post #9

Earlier quoted context omitted.

I know very little about how the environment where they run these models look, but surely they have access to different tools like vector embeddings with more current data on various topics?

If they could "see" the future and exploit that they'd probably have much higher returns.

I would say that if these models independently could create such high returns all these companies would shut down the external access to the models and just have their own money making machine. :)

Re: We gave 5 LLMs $100K to trade stocks for 8 months

#194

Post by plufz »

disconcision wrote: post #53
plufz wrote: post #9

Earlier quoted context omitted.

I know very little about how the environment where they run these models look, but surely they have access to different tools like vector embeddings with more current data on various topics?

you can (via the api, or to a lesser degree through the setting in the web client) determine what tools if any a model can use

But isn’t that more which MCP:s you can configure it to use? Do we have any idea which secret sauce stuff they have? Surely it’s not just a raw model that they are executing?

Re: We gave 5 LLMs $100K to trade stocks for 8 months

#196

Post by doe88 »

bdangubic wrote: post #99
nickthegreek wrote: post #89

Earlier quoted context omitted.

My understanding is that grok api is way different than the grok x bot. Which of course does Grok as a business any favors. Personally, I do not engage with either.

you gotta be quite a crazy person to use grok :)

Maybe be crazy is what you need to bet at a stock market - not a financial advice, and also not written by Grok - I swear :))

Re: We gave 5 LLMs $100K to trade stocks for 8 months

#197

Post by rcarmo »

I spent a while looking at trading algos a few years back (partly because of quant stuff I got involved in, and partly out of curiosity). I found that none of the “slow” trading (i.e., that you could run at home alongside your day trading account) was substantially effective (at least in my sampling), but I never thought an LLM would be any good at it because all the analysis is quantitative, not qualitative or contextual.

In short, I don’t think this study proves anything unless they gave the LLMs additional context besides the pure trading data (Bloomberg terminals have news for a reason—there’s typically a lot more context in he market than individual stock values or history).

Re: We gave 5 LLMs $100K to trade stocks for 8 months

#198

Post by gwd »

turtletontine wrote: post #49
gwd wrote: post #14

The summary to me is here: > Almost all the models had a tech-heavy portfolio which led them to do well. Gemini ended up in last place since it was the only one that had a large portfolio of non-tech stocks. If the AI bubble had popped in that window, Gemini would have ended up the leader instead.

Yup. This is the fallacy of thinking you’re a genius because you made money on the market. Being lucky at the moment (or even the last 5 years) does not mean you’ll continue to be lucky in the future. “Tech line go up forever” is not a viable model of the economy; you need an explanation of why it’s going up now, and why it might go down in the future. And also models of many other industries, to understand when and…

What would have been impressive is if the favored industries, or individual companies, experienced a major drop during the target testing window, and the LLMs managed to pull out of those industries before they dropped.

Re: We gave 5 LLMs $100K to trade stocks for 8 months

#199

Post by ta12653421 »

itake wrote: post #91

Model output is non-deterministic. Did they make 10 calls per decision and then choose the majority? or did they just recreate the monkey picking stocks strategy?

++1

This.

Thats also the reason why i still belive in "classic instruments" when configuring my trade app; the model wont give you the same entries on lets say 5 questions.

Post reply on HN