Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLMs can teach themselves to better predict the future

arxiv.org

Post reply on HN
Print view View original

31–40 of 92 posts

Re: LLMs can teach themselves to better predict the future

#31

Post by bturtel »

nyrikki wrote: post #12

While interesting, the title is obviously a bit misleading. > Our results on a temporally held-out test set of questions resolving after December 25, 2024 show that for both of the models that we employed our method on, Phi-4 14B [15] and DeepSeek-R1 14B [14], we find accuracy improvements of between 7–10% over the base versions of these models as well as the same models fine-tuned with randomized outcome labels as a…

We're working on a follow up paper now to show similar results with larger models!

Re: LLMs can teach themselves to better predict the future

#32

Post by pizza »

huijzer wrote: post #30

Makes sense. Renaissance Technologies used machine learning to get an annual return of around 60% for multiple years even when they had large piles of money already. They already showed that machine learning can predict the future.

I got the impression from somewhere that they used the simplest machine learning techniques (just fitting regressions to data), but that it was "the 'what' that they decided to fit" that was the secret sauce.

Re: LLMs can teach themselves to better predict the future

#33

Post by artembugara »

bguberfain wrote: post #29
dantheman252 wrote: post #2

Danny here, one of the authors of this paper. If anyone has any questions or anything feel free to AMA!

Any chance you could release the dataset to the public? I imagine NewsCatcher and Polymarket might not agree..

Co-founder of NewsCatcher (YC S22). There are some reasons for not having a dataset fully open sourced.

But we have free/very very low tiers for academia.

So in case you need access for your research, go to https://www.newscatcherapi.com/free-news-api

Or feel free to email me directly at artem@newscatcherapi.com

Re: LLMs can teach themselves to better predict the future

#34

Post by psychoslave »

kelseyfrog wrote: post #7
4b11b4 wrote: post #6

but is it really reasoning? honest question re the underlying architecture of transformers also, self play seems quite an intuitive approach. There's another interesting paper from deep mind about play

You can call it blorbblorb if it makes you feel better. Reasoning is a social construct which, for many people, is grounded in humanity. Others ground it using other socially transmitted ontologies. We don't usually discuss how people choose to ground their ontological beliefs, but why not? Why did you choose to ground "reasoning" in the way you do? If you didn't choose, why not?

To start with, "I/you" is most of the time a meaningless or at best very ambigous term.

Let's say that here "I" is taken as synonym of "the present reflective attention".

Can the question "did I chose to ground reasoning?" in such a context be attached to a meaningful interpretation? And if so, is the answer reachable by the means available to "I"? Can "I" transcend "my" beliefs through contemplation of "my" own affabulations?

Re: LLMs can teach themselves to better predict the future

#35

Post by artembugara »

Artem here, co-founder of NewsCatcher (YC S22), our data has been used for research.

Danny and team our old friends who are using our free/super-low pricing for academia and researchers.

AMA, or feel free to email artem@newscatcherapi.com

https://www.newscatcherapi.com/free-news-api

Re: LLMs can teach themselves to better predict the future

#36

Post by psychoslave »

kelseyfrog wrote: post #18
ttpphd wrote: post #14

Earlier quoted context omitted.

Throwing your hands up in the air like this doesn't help build a constructive case for using the word reasoning. It builds a case that words mean whatever

Yes, words mean whatever. See Saussure and Wittgenstein. To advance the claim that words are objective is to confuse the symbolic with the real. This is generally regarded by engineer-types as false, but societal taboos and power structures can be revealed by noting what speech provokes the strongest reactions.

Saussure didn't use"arbitrary" in the sense "with absolutely unrestricted selection of signifiant/signifié association regardless of the context."

I'm not sure what links you try to show and what you try to argue here though.

Re: LLMs can teach themselves to better predict the future

#37

Post by empath75 »

There are two ways you can get better at predicting the future. One is the obvious one of being really good at discerning signals.

The other way is to alter the future to match your predictions.

This is something to think about when you combine something like this kind of training with agentic workflows.

Re: LLMs can teach themselves to better predict the future

#38

Post by empath75 »

jddj wrote: post #13
matthest wrote: post #9

Earlier quoted context omitted.

Assuming LLMs eventually get really really good at this. Do you see this destroying prediction-based markets (i.e. the stock market and Polymarket)? Markets exist because there's uncertainty about the future. If LLMs can predict with extremely high accuracy, would there no longer be a need for markets?

If your oracle can tell me (and everyone else) the prevailing price of copper in 6 months in a manner which accounts for the reflexivity of everyone suddenly learning what will be the precise prevailing price of copper in 6 months, you've got yourself a perfect universe simulator and I'm not sure what the point is of worrying about any hypotheticals (or copper) at that point.

If one developed such an oracle, you would surely not share it.

Re: LLMs can teach themselves to better predict the future

#39

Post by psychoslave »

kelseyfrog wrote: post #21
ttpphd wrote: post #20

Earlier quoted context omitted.

"societal taboos and power structures can be revealed by noting what speech provokes the strongest reactions" Ok I'll bite. Who is the marginalized Other?

It's taboo to believe that LLMs can reason. People who believe this are systematically de-legitimized and framed as being out of or at least out of touch with reality. This will appear as common sense or naturally true if you're inside the LLMs-cant-reason ideology.

It's not taboo, it's just ridiculous given the state of the art.

That doesn't mean that a silicon based reasoning entity is an ontological impossibility. But if it is to become a reality, it's not necessarily through LLM that such an entity will be spawn.

Re: LLMs can teach themselves to better predict the future

#40

Post by dantheman252 »

artembugara wrote: post #35

Artem here, co-founder of NewsCatcher (YC S22), our data has been used for research. Danny and team our old friends who are using our free/super-low pricing for academia and researchers. AMA, or feel free to email artem@newscatcherapi.com https://www.newscatcherapi.com/free-news-api

Hey Artem, NewsCatcher has been a great resource in our news pipelines!
Post reply on HN