just now
Kalshi Research has published a working paper examining whether prices on the prediction market can be read literally as probabilities, drawing on 2,243,741 resolved markets across eleven categories from the platform's launch in 2021 through mid-2026. The paper, "Calibration in Prediction Markets: Theory and Evidence", is authored by Nicole Kagan and Rubens Baiocchi and dated August 2026.
The study assesses calibration, which compares the price quoted on a contract with the frequency at which such contracts resolve yes, alongside two accuracy measures. Calibration is quantified using the Brier score, a proper scoring rule originally developed for meteorological forecasting, where zero is perfect, 0.25 corresponds to always guessing 50 per cent against a 50/50 base rate, and one is the worst possible outcome. The paper notes that the most skilled superforecasters typically record Brier scores of around 0.1 to 0.15.
Across all markets excluding Exotics, the study records a decline in Brier score from approximately 0.087 at a three-month horizon to below 0.02 at market close. Naive accuracy, defined as whether the higher-priced side of a contract turned out to be correct, rises from 88.3 per cent at three months to 97.2 per cent at close.
Nicole Kagan and Rubens Baiocchi, Authors, Kalshi Research:
We interpret these findings, taken together, as variation around a strong baseline rather than as a qualification of it: Kalshi's prices behave like genuine probabilities, and increasingly so as resolution approaches.
The authors present results under two constructions. The per-horizon measure includes every contract with a price observation at a given horizon, while the fixed-cohort measure restricts every horizon to the same 35,562 markets that existed a full month before resolution. The distinction matters most at the one-day horizon, where the per-horizon Brier score of 0.126 reflects roughly 492,000 short-fused daily, hourly and sub-hourly markets, predominantly in sports and crypto, that exist only in the final day before resolution. The fixed-cohort figure for the same longer-dated markets observed one day out is 0.039.
Removing Sports, Mentions and purely intraday markets eliminates the anomaly. On that subsample, Brier scores decline monotonically from 0.088 at three months to 0.030 at close on a per-horizon basis, and to 0.016 on the fixed cohort.
The study also tests how much of the aggregate result is carried by markets already close to certain. A tail-truncated version retaining only markets priced above 2 and below 98 cents produces higher Brier scores at every horizon, at 0.077 versus 0.030 at close and 0.105 versus 0.088 at three months. The authors argue that pricing an event at 99 per cent or 1 per cent requires real information rather than being a statistical artifact, and that the truncated figures remain well below the 0.25 a coin-flip forecaster would earn.
By category, Elections record the lowest Brier scores, at 0.004 at close, followed by Mentions at 0.007 and Climate at 0.013. Economics markets show the smoothest profile, declining almost linearly from 0.108 at three months to 0.066 at close, a pattern the paper links to their resolution against pre-scheduled statistical releases such as CPI prints, payrolls reports and GDP estimates. The weakest cell reported in the category table is Mentions at the two-month horizon, at 0.192, which remains below the 0.25 randomness benchmark.
Calibration improves with both dollar volume and the number of unique traders. At market close, markets with less than $10,000 in total event volume record a Brier score of 0.0635, against 0.0099 for those above $200,000. Across trader-count buckets at close, the figure moves from 0.0354 for markets with fewer than 20 traders to 0.0089 for those with at least 1,000. One-day markets cross the 0.05 Brier-score threshold at 20 traders, one-week markets at 901 traders, and one-month markets only at approximately 1,900 traders.
Nicole Kagan and Rubens Baiocchi, Authors, Kalshi Research:
Depth of participation, in other words, improves calibration, but only up to a limit set by how much genuine uncertainty remains at a given horizon: no amount of additional trading fully substitutes for the passage of time.
The relationship reverses at the longest horizon. At three months, markets with at least 1,000 traders are slightly worse calibrated, at 0.0929, than markets with fewer than 20 traders, at 0.0855. The paper attributes this to a confound between participation and event salience, on the basis that markets attracting a thousand traders three months ahead of resolution tend to be national elections and championship games, which are intrinsically harder to forecast than a low-attention long-dated market.
A further section re-anchors the resolution clock from the timestamp at which Kalshi formally closes a market to the moment the underlying event actually occurred. Using a proprietary dataset of to-the-second game-end timestamps, the authors find Sports calibration tracks the 45 degree line closely at the one-week, one-day and one-hour horizons, with concentration at the tails in the instant a game ends, which they attribute to an information-propagation lag and to thinly traded bracket markets vulnerable to stale pricing.
For Elections, the effect runs the other way. Anchoring to election day rather than market close raises the one-month Brier score from 0.045 to 0.079, and to 0.076 for markets asking specifically which candidate wins a given race. The paper notes that Election-category markets were contractually unable to resolve earlier than vote certification, or in some cases inauguration, until November 2025.
Nicole Kagan and Rubens Baiocchi, Authors, Kalshi Research:
Though novel here, we believe that this approach represents a more faithful interpretation of Election calibration and hope for it to become the new industry reporting standard.
On lead-market accuracy, which asks whether the single most-favoured outcome among several mutually exclusive alternatives occurred, the full sample ranges between approximately 57 per cent and 66 per cent from three months through one day before converging on per-market accuracy levels by the one-hour and close horizons. The filtered sample runs at approximately 63 to 69 per cent at long-to-medium horizons, reaching the low 80s by close. The authors argue the relevant benchmark is the true dispersion of outcomes in a given race rather than 100 per cent, since a well-calibrated market should see a candidate priced at 60 cents win about 60 per cent of the time.
Kalshi received approval from the US Commodity Futures Trading Commission in November 2020 as a Designated Contract Market, the same regulatory category governing traditional futures exchanges such as the Chicago Mercantile Exchange, and began operating in 2021. Its core product is the event contract, an Arrow-Debreu security paying $1 if a specified outcome occurs and $0 otherwise, quoted in cents between 1 and 99 and matched between a maker and a taker.
The paper situates its findings against an existing literature that includes Berg, Nelson and Rietz (2007) on the Iowa Electronic Markets, Cowgill and Zitzewitz (2015) on corporate prediction markets at Google and Ford, and Diercks, Katz and Wright (2026) on Kalshi's macroeconomic markets. It also acknowledges a documented favourite-longshot bias in prediction market prices, which it sets aside as outside its scope.
The authors set out several limitations. The relationships between calibration and both volume and trader count are described as correlational rather than causal, since higher-profile events plausibly attract more participants and better information independently. The 2 to 98 cent truncation threshold is described as defensible but discretionary, and the date-truncation exercise was applied only to categories where matching time data was available. The naive and lead-market accuracy measures are binary criteria that do not account for how confident a given price was, and the authors indicate that a conviction-weighted accuracy measure will follow in future work from Kalshi Research.
With the exception of trader count, the underlying data is public and reproducible via the Kalshi API.
Found this interesting? Become a member of LiquidityFinder and get daily industry news direct to your inbox — join here.
