Decision market challenges
Prediction markets are an increasingly popular primitive, currently being used to facilitate speculation on the probability of various political, geopolitical and economic events. While this use case is promising, as demonstrated by the increasingly high TVL of protocols such as Polymarket, it only unlocks a fraction of what prediction markets have to offer.
Decision markets are how prediction markets become useful for making decisions. The event probabilities which prediction markets currently elicit can sometimes be useful for informing decisions, however they are not optimised for this purpose.
Prediction markets only reveal the probability of an event, whereas decision markets reveal how different actions would impact the outcome being predicted. For example, a prediction market might reveal that p(GDP increasing year over year) = 0.4 (where p() denotes probability), whereas a decision market could tell us that conditional on electing Ron Paul, p(GDP increasing) = 0.9, whereas only 0.06 if Kamala Harris were to be elected (figures are for illustrative purposes only).
While the output of a prediction market can be useful, the conditional probabilities provided by conditional prediction markets are much more actionable. In this case, they could be used to conclude that voting for Ron Paul is the superior action, if one’s goal is a higher GDP.
All the above however is still framed in the context of politics, which despite being the domain in which prediction markets have excelled, is unlikely to be where decision markets create the most value. Politics-focused prediction markets are popular because people enjoy betting on matters which they have tribal or ideological connections to, given that the target customer base of prediction markets, such as Polymarket, is speculators. The target customer base of decision markets, however, will be information buyers; those willing to pay to discover which action the market believes they should take in order to maximise success according to a given metric.
The goal of decision markets is to elicit from market participants what the expected impact of an action is on a selected metric. While this might seem simple, given the below high level outline, there are several confounding, alignment and manipulation risks which are important to consider and potentially mitigate and which are the focus of this article.
Mechanism overview
Here’s how decision markets work in the context of a DAO evaluating funding proposals (where funding a proposal is considered an action):
For each funding action, we create two prediction markets:
- A “Yes” market that predicts the outcome if the action is taken
- A “No” market that predicts the outcome if the action is not taken
These markets use a scalar design where the payout of tokens is proportional to the value of a composite metric. The key tokens to understand are:
- $\textsf{Long}^{\text{yes}}$: Redeems for a share value proportional to the metric’s value if the action is taken
- $\textsf{Long}^{\text{no}}$: Redeems for a share value proportional to the metric’s value if the action is not taken
The relative price of $\textsf{Long}^{\text{no}}$ and $\textsf{Long}^{\text{yes}}$ reveal what the market expects each action’s impact to be on the selected metric (subject to caveats, analysis of which will constitute the remainder of this report). These prices can be used to select which actions to take, on the basis of the ratio between each action’s cost and its predicted impact, i.e. its cost:benefit ratio.
Implementation of the above requires selection of an AMM design or other mechanism for facilitating the exchange of shares and construction of the composite metric on which to evaluate actions. A decision rule also must be selected, which is an algorithm for deciding which actions to take, given the predicted impact on the metric and magnitude of each action’s respective cost.
For some further reading on decision market design, see:
- https://community.ggresear.ch/t/conditional-funding-markets/27
- https://github.com/zack-bitcoin/amoveo-docs/blob/3747c9c7b330a03531da0cbeb111fafac6839f81/basics/msrs_and_prediction_markets.md
- https://ethresear.ch/t/prediction-market-design-for-betting-on-many-highly-improbable-events/8280
- https://ethresear.ch/t/practical-futarchy-setup/10339
- https://timroughgarden.github.io/fob21/reports/ZLRL.pdf
- https://ethresear.ch/t/possible-futarchy-setups/1820
- https://mason.gmu.edu/~rhanson/combobet.pdf
- https://github.com/metaDAOproject/Manifesto/blob/main/Manifesto.pdf
Confounding
Decision markets can suffer from confounding because they are trying to use market-implied conditional probabilities as a proxy for measuring the impact of an action on the value of a metric. Decisions are made based on the relative prices of $\textsf{Long}^{\text{yes}}$ and $\textsf{Long}^{\text{no}}$ shares, in each action’s market. Any factors which speculators consider when pricing these shares, aside from their expectations of an action’s impact, are confounding variables. Confounding variables distort the price with information which is irrelevant to maximising the value of the metric, hence making the decision market prices less useful.
Confounding occurs when the variable you are measuring the impact on (the dependent variable) is affected by variables other than the variable you are actually trying to measure the impact of (the independent variable). You might want to measure the effect of A on B but end up accidentally measuring two types of confounds: hidden variable confounds (the effect that C has on both A and B) or reverse causality confounds (the effect of B on A). An illustration of third variable confounding is that ice cream sales are correlated with shark attacks, but one should be careful not to conclude from this that recently having ingested ice cream causes sharks to pay them special attention.
Higher consumption of ice cream
↑
Warmer weather <———- confounding third variable
↓
More people swimming at the beach
↓
More shark attacks
An equivalent example in the context of reverse causality would be concluding that painkillers cause injuries due to consumption of painkillers and injuries being highly correlated.
Injury
↓ <———- confounding reverse causal pathway
Consumption of painkillers
Strategy: Standard futarchy setup
The most straightforward setup is to base action decisions directly on the decision market’s output and decision rule. This approach is typically what people mean when they refer to “decision markets.” It has two major advantages:
- Decisions are informed by the information generated by the prediction markets.
- Hidden variable confounds and reverse causality are mitigated
At a high level, this setup achieves these benefits by forcing all potential confounds through a single, narrow bottleneck: \frac{\textsf{Long}^{\text{yes}}}{\textsf{Long}^{\text{no}}}. \frac{\textsf{Long}^{\text{yes}}}{\textsf{Long}^{\text{no}}} represents the decision market’s expectation of the impact of taking vs not taking an action, on the value of the metric. From here onwards, \frac{\textsf{Long}^{\text{yes}}}{\textsf{Long}^{\text{no}}} will be abbreviated as score(action), where a higher score(action) indicates an action is assessed as being more favourable.
Third variable confounding and reverse causality are only possible if the choice of action is influenced by a confounding variable. As a result, the relevance of score(action) to confounding is hard to overstate, as any confounding variable must go through score(action) in order to influence the choice of action, given that actions are chosen according to their respective score(action) values. score(action) is the sole thing standing between the standard futarchy setup and one where confounding is impossible, due to e.g. actions being selected at random.
Below are two causal diagrams which depict a hypothetical example of confounding in the standard futarchy setup.
Third variable confounding:
action is taken
↑
score(action)
↑
Hypothetical confounding variable
↓
metric
Reverse causality:
metric
↓ <——– hypothetical reverse-causal pathway, causing confounding
score(action)
↓
action is taken
Third variable confounding
Third variable confounding is possible in this setup if a variable exists which influences both choice of action
and the value of metric
, changing their statistical relationship, as depicted in the above diagram. This is because doing so would distort score(action)
, which effectively measures the statistical relationship between metric
and choice of action
. We do not need to look far to find such a variable, as score(action)
itself is one! Check the above causal diagrams to verify that score(action) influences both metric and choice of action.
An action is more likely to be taken if decision market participants expect it to be beneficial, hence its expected impact on the metric, conditional on being chosen, is biased upwards. Phrased differently, EV(metric | action is taken) positively influences both p(action is taken) and EV(metric), hence makes them artificially positively correlated, increasing the value of EV(metric | action is taken), which, in a sense, measures their correlation. |