[dead]
Looking at the data behind prediction markets
41–50 of 55 posts
Re: Looking at the data behind prediction markets
#42I think a fundamental problem is that the customer of a prediction market is the trader (gambler?), not the public. If you want accurate forecasts, you need sharp traders. If you want sharp traders, you need to pay them a lot. As a platform, the straightforward way to do that is to attract a large number of uninformed gamblers. And ultimately, the accuracy is not determined by volume, but by the fraction of informed and uninformed capital that is trading for idiosyncratic reasons uncorrelated to the "true" probability. Someone has to put in the effort to make the markets accurate, and that someone has to be paid and that money has to come from somewhere.
Re: Looking at the data behind prediction markets
#43I think a fundamental problem is that the customer of a prediction market is the trader (gambler?), not the public. If you want accurate forecasts, you need sharp traders. If you want sharp traders, you need to pay them a lot. As a platform, the straightforward way to do that is to attract a large number of uninformed gamblers. And ultimately, the accuracy is not determined by volume, but by the fraction of informed…
> Someone has to put in the effort to make the markets accurate, and that someone has to be paid and that money has to come from somewhere.
The foundational idea of prediction markets is that this payment comes from the market itself. If you have a market full of suckers, the experts with real knowledge are incentivized to participate in the market to profit from the gap between what the market forecasts and their own expert forecast. This in turn will drive the market's prediction to be more accurate by incorporating the expert knowledge in directly form of the their "bets". In effect the market says put up or shut up to everyone who thinks they know better the market.
Viewed like this, prediction markets aren't much different from stock markets that also work by the premise of (active) investors claiming to know better than the market. It all follows from the efficient-market hypothesis.
All this said, I find the trend of prediction markets being used to predict what are effectively mostly random outcomes seems a bit strange. But on the other hand, the mere existence of such markets does provide financial incentive to come up with new, better ways to predict these outcomes. This itself would be very useful, at least as long as the subject is more akin to predicting the weather than the movements of a football.
Ultimately, I'm still an optimist when it comes to prediction markets.
Re: Looking at the data behind prediction markets
#44It sounds like they should be called "indicator markets" rather than "prediction markets", as the data shows they largely just summarize the current knowledge, with little predictive ability.
I don't understand the distinction you are making. Obviously they are based on current knowledge. Nobody has any actual crystal ball. But the outcomes are with regard to future events. So the correct term is predictions. And they don't "just summarize the current knowledge". The whole point is that they better reflect the knowledge of people who presumably know better because they are willing to put their money where…
My sense is that, for prediction markets to work, there needs to be some real knowledge/analysis/judgement spread across at least a material subset of participants. Simply aggregating random guesses is likely no better than any given random guess.
Put another way there needs to be SOME signal buried in all the noise.
Re: Looking at the data behind prediction markets
#45Earlier quoted context omitted.
I don't understand the distinction you are making. Obviously they are based on current knowledge. Nobody has any actual crystal ball. But the outcomes are with regard to future events. So the correct term is predictions. And they don't "just summarize the current knowledge". The whole point is that they better reflect the knowledge of people who presumably know better because they are willing to put their money where…
My sense is that, for prediction markets to work, there needs to be some real knowledge/analysis/judgement spread across at least a material subset of participants. Simply aggregating random guesses is likely no better than any given random guess. Put another way there needs to be SOME signal buried in all the noise.
And there is. That's why they work. Prediction markets are not simply aggregating random guesses. Were you somehow under the impression that they were?
Re: Looking at the data behind prediction markets
#46“It’s clear that Polymarket and Kalshi host these markets to serve bettors, not to produce useful information. Take Kalshi’s “Best AIs this week?” markets, which not only cover too short a time period to be useful in any decision.”
Yeah the site is clearly optimized for betting at the expense of the purported purpose.
Looking at polymarket, wagers that could yield interesting odds are generally turned into meaningless junk by contrived timelines or other technicalities that aren’t at all related to the core thesis of the wager. I.e the flashy thing the wager talked about wasn’t the main contention of the odds, an artificially constrained timeline or other not useful technical detail is what the wagers hinge on.
It’s also strange how so few had advance cutoffs. The person with “info” is incentivized to hold their bet to the last minute, so even accepting the premise that the markets provide info, the wagers seem structurally designed to give that info when’s it’s too late to be of use. If the wanted to fix that it would be easy but they don’t. It’s as simple as forcing long run wagers to have a gap between the wager period and the resolution time point.
Re: Looking at the data behind prediction markets
#47From a technical perspective how does one start to work with this historic prediction market data effectively?
Where do you source it from? What tools are most effective?
I’d love to see a more technical article on how that would work as someone who isn’t a data scientist.
I looked on Hugging Face and saw there was a 163GB historical Polymarket dataset. Is that a good place to start?
Re: Looking at the data behind prediction markets
#48One thing that really jumps out to me is the lack of a performance gap between the 90-day and 30-day resolution times. If 2-months of new information doesn't lead to materially improved forecast, then to me this seems to strongly reinforce the takeaway that these markets aren't really forecasting, so much as "the oracle is largely saying what other oracles already say, just updated faster." Am I misunderstanding the…
On the other hand, both 90 days off and 30 days off are both still a very long time before an event happens.
It's going to highly depend on the type of event. But it's not surprising to me that there will often not be much difference, because the main factors affecting the event might not really start to be known until just a few days in advance.
Which really makes me wonder about in which categories our prediction markets most useful at a timeframe of months, versus which categories are most useful at a timeframe of days.
Re: Looking at the data behind prediction markets
#49Earlier quoted context omitted.
Its based on high quantity decision making, and quantity is a sort of quality if you squint and turn your head
"Quantity has a quality all its own" —known economic genius, Joseph Stalin
I mean, someone can have a terribly bad ideology and still be right sometimes. In fact, that's almost always the case.
Re: Looking at the data behind prediction markets
#50Random aside: I distinctly remember getting on a phone call with people from the SEC (US Gov't) with the goal of understanding if I could legally start a prediction market. This was during 2020 or 2021. I recall them saying basically "no way" and that it wouldn't be legal, and would be rife with abuse. Fun times.
Random aside: Prediction markets are illegal to be run under SEC jurisdiction. The CFTC (a different agency) allows and oversees them under their jurisdiction.
You called the wrong agency.