Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The AI Superforecasters Are Here

astralcodexten.com

Post reply on HN
Print view View original

51–60 of 60 posts

Re: The AI Superforecasters Are Here

#51

Post by ddp26 »

mikgp wrote: post #40

It feels like Scott is taking the bull case here - but like “perfect” information will pervert markets in bizarre ways. As soon as the prediction market says “this path here has a 25% chance of curing cancer” all sorts of money is moved away from other things. It will absolutely cause political outcomes not just predict them. And then of course there’s the cheating element. Anything that’s feasible to change the outc…

But Scott's point is more: why even have markets? Once you have the superforecasting available on the questions you care about, why do you need to publish it for everyone to also react to?

Re: The AI Superforecasters Are Here

#52

Post by mikgp »

ddp26 wrote: post #51
mikgp wrote: post #40

It feels like Scott is taking the bull case here - but like “perfect” information will pervert markets in bizarre ways. As soon as the prediction market says “this path here has a 25% chance of curing cancer” all sorts of money is moved away from other things. It will absolutely cause political outcomes not just predict them. And then of course there’s the cheating element. Anything that’s feasible to change the outc…

But Scott's point is more: why even have markets? Once you have the superforecasting available on the questions you care about, why do you need to publish it for everyone to also react to?

I mean he explicitly at the end says there will be a new era of prediction markets:

"This, then, is my prediction for the AI superforecaster future: for basic questions, your off-the-shelf AI chatbot will be able to offer opinionated probabilities superior to those of any human. For more controversial or bias-laden questions, a new era of prediction markets will smooth over differences in brand and model and efficiently aggregate all AIs’ opinions."

But one reason that you would publish it for everyone else to react was exactly what I said, because if you can show people that a superhuman AI believes a certain outcome is more likely, you impact whether or not that certain outcome is more likely. Which feels like a flaw in this version of the future. That in fact you could overcorrect and make the markets less efficient.

Re: The AI Superforecasters Are Here

#53

Post by danaris »

chucksmash wrote: post #44
kennywinker wrote: post #32

Earlier quoted context omitted.

> I met an AI superforecaster startup founder who told me his AI had turned $35 into $2 million on Kalshi over seven months Ok… assuming you can’t use that $2mil for some reason, simply take out a $10k line of credit and you’ll have $571 million in 7 months. If you have the 2mil, congrats you’re 7 months away from $114B - you’re now one of the top 20 richest people in the world. If this was truely the money printing…

Any given trade also has a capacity. A mispricing opportunity can only absorb so much investment before it's no longer mispriced. A particular trade that can 2x $20k won't be able to do the same for $20 billion. It's why RenTech capped their Medallion Fund and closed it to outside investment. If there's only a billion dollars sloshing around on Kalshi, you can't expect to put $1 trillion into bets and take $2 trillio…

But if these "superforecasters" can forecast just about anything accurately—which seems to be the claim—then they're not limited to any one claim, or even a small subset of them. They can place bets on everything and expect to win most of the time.

This doesn't pass the smell test.

Re: The AI Superforecasters Are Here

#54

Post by jimbokun »

exe34 wrote: post #6
glimshe wrote: post #5

Earlier quoted context omitted.

You got it (I was going to say "nailed it" but that's becoming an LLM marker). This is exactly what I feel about a lot of the paid investment advice out there. Compounding can make any decent alpha worth a ton, to the point that these people would be investment bankers and not advisors if they knew what they were talking about.

Same with get rich quick books.

I realized paid content creation wasn’t a good business when the only successful paid content creation was courses teaching you how to make money from creating paid content.

Re: The AI Superforecasters Are Here

#55

Post by jimbokun »

gwern wrote: post #20
datadrivenangel wrote: post #2

If the forecasting models were so good that people were actually consistently beating prediction markets, they wouldn't be starting startups to be selling it. And even if it is good enough, once you're shelling out thousands of dollars a year in research costs, does that give you any remaining alpha?

> And even if it is good enough, once you're shelling out thousands of dollars a year in research costs, does that give you any remaining alpha? That's precisely why you would want to make a startup to get investment now rather than self-fund and bootstrap. That alpha isn't going to last forever, especially because everyone has access to the frontier LLMs, which keep getting better, and will eventually beat your fanc…

The article covers the purpose for prediction markets after super human predictors: to deal with bias or ulterior motives trained into the models.

Re: The AI Superforecasters Are Here

#56

Post by [deleted] »

kennywinker wrote: post #32
gwern wrote: post #20

Earlier quoted context omitted.

> And even if it is good enough, once you're shelling out thousands of dollars a year in research costs, does that give you any remaining alpha? That's precisely why you would want to make a startup to get investment now rather than self-fund and bootstrap. That alpha isn't going to last forever, especially because everyone has access to the frontier LLMs, which keep getting better, and will eventually beat your fanc…

> I met an AI superforecaster startup founder who told me his AI had turned $35 into $2 million on Kalshi over seven months Ok… assuming you can’t use that $2mil for some reason, simply take out a $10k line of credit and you’ll have $571 million in 7 months. If you have the 2mil, congrats you’re 7 months away from $114B - you’re now one of the top 20 richest people in the world. If this was truely the money printing…

[deleted]

Re: The AI Superforecasters Are Here

#57

Post by gwern »

kennywinker wrote: post #48
gwern wrote: post #38

Earlier quoted context omitted.

You're attacking a strawman. No one is claiming that you can pull off that multiplier at arbitrary amounts arbitrary amounts of times. And 7 months is plenty of calendar time for those arbitrages to disappear, given the attention on the area and the rapid rate of development. (Warren Buffett can't pull off his early trades now either, doesn't mean he was stupid or grifting in taking early investment.)

So you agree this stuff isn’t useful?

Wow, I obviously do not agree with that, and since you're trying to put words in my mouth and false dichotomies, I think that's the last question of yours I will be answering.

Re: The AI Superforecasters Are Here

#58

Post by ffitch »

kennywinker wrote: post #47
ffitch wrote: post #39

Earlier quoted context omitted.

from the article: I asked the guy who turned $35 into $2 million in seven months on Kalshi whether, in another seven months, he would be able to 100,000x his money a second time to $200 billion. Unsurprisingly, he said no - there’s only so much easy money on Kalshi, and his AI had already taken it all (also, other people with similar AIs are starting to fight him for it!)

So this stuff is only as useful with a steady supply of people making less intelligent bets. You don’t have to sell me on it being bullshit, but you are.

The article is not about this stuff being useful for making money off of prediction markets.

Re: The AI Superforecasters Are Here

#59

Post by kennywinker »

gwern wrote: post #57
kennywinker wrote: post #48

Earlier quoted context omitted.

So you agree this stuff isn’t useful?

Wow, I obviously do not agree with that, and since you're trying to put words in my mouth and false dichotomies, I think that's the last question of yours I will be answering.

I was specifically responding to your comment by pointing out that you’re describing something that is valuable now but will be of less and less value as the market adapts. If you think that’s a good investment, that’s your business, of course.

Re: The AI Superforecasters Are Here

#60

Post by mikgp »

There’s something of a footnote in here that - raises a moral or even spiritual question.

Scott says that one of the great things about superforcasters is that some people who regularly make bad decisions will be able to make better decisions by asking superforcasters questions like “Is this marriage likely to end in divorce.”

And there’s a number of profound questions in here about - what are we all here for?

There’s something of a philosophical horseshoe theory here which is like - I might argue - this sounds like astrology or other metaphysical forms of divination.

And the counter argument would be something like - no it’s completely different because this is true. Which is of course what all the spiritual traditions say about each other.

So it begs the question - is it the solution or the process.

Is using a super forecaster better because it’s “real”; or like I think I could make the argument it’s worse for that reason. Asking your grandmother or spiritual advisor is part of a process of existence. You ask multiple people, you have the experience, maybe you marry the wrong person, you learn and you grow.

But when you outsource that decision making.

Post reply on HN