Transparent Randomization

A Field Study of the Champions League

Alistair Wilson

University of Pittsburgh

Marta Boczon

Copenhagen Business School

Main Idea

  • Randomization a frequently used market design tool:
    • Often used to ensure fair treatment 
    • Establishes fair treatment ex ante, where outcomes can still be very unfair
  • But if the point of randomization is equity:
    • not only does it need to actually be  fair
    • It also needs to be perceived/understood as fair
  • But this can be hard in complicated mechanisms, or constrained assignments where outcomes are complex and inter-related
    • How to have stakeholders understand their ex ante fair treatment!?

Solutions

Will outline a high-stakes randomized assignment with non-trivial constraints that has been broadly accepted by the general public with the following elements:

  • Physically conducted draws 
  • All randomizations as simple uniform draws
  • Dynamic procedure that assembles the assignment with comprehensible elements
  • Separation between randomization and algorithm
    • Algorithm: Deterministic (but very complex)
    • Randomization: Simple draws from an urn

Most Public-Facing Randomizations are Constructed with Uniforms

Most Public-Facing Randomizations are Constructed with Uniforms

But examples make clear some design latitude:

  • While component are discrete uniforms...
    • ... distributions often non-trivial interactions 
    • ... both with/without replacement

UEFA Champions League Draw

  • High stakes (millions of euro on the line)
  • Huge public engagement (millions of viewers)
  • Very complicated outcome:
    • Matching of 8 teams to 8 teams
    • Non-trivial matching constraints
      • Teams that played together at prior stage
      • Teams from the same country/association

Urn \(A\)

Urn \(B\)

Procedure

Urn \(A\)

Urn \(B\)

Round 1: Runner-up

Urn \(A\)

Urn \(B\)

Round 1: Runner-up

Excluded

Urn \(A\)

Urn \(B\)

Round 1: Winner

Excluded

Urn \(A\)

Urn \(B\)

Match made

Excluded

Urn \(A\)

Urn \(B\)

Round 2: Runner Up

Excluded

Urn \(A\)

Urn \(B\)

Round 2: Runner Up

Excluded

Urn \(A\)

Urn \(B\)

Round 2: Winner

Excluded

Urn \(A\)

Urn \(B\)

And so on...

Excluded

Dynamic Urn Draw

  • Constraint set \(H\) implies set of feasible assignments \(\Gamma_H\)
  • Direct constraint make sure that excluded team pairs aren't allowed to happen
  • Dynamic constraints via a computer algorithm ensure that given the current state of the draw, all subsequent draws will not violate the constraints:
W(r,\mu )=\left\{w\in\mathcal{W} \left| \exists\left(\mu,\mu^{\prime} \right) \in \Gamma_H \text{ s.t. }(w,r)\in\mu^{\prime} \right.\right\}

Winners

Exists feasible assignment given current state \(\mu\)

\(r\) is valid partner

Resulting Match Probabilities

(2018 Draw)

  • Can we quantify the distortions across otherwise equally treated teams?
  • Can we construct a fairer randomization?

Distortions

Distortions

Distortions

  • Largest beneficiary: Real Madrid in 2017
    • Realization: Champions
  • Worst hit: Arsenal in 2014
    • Realization: eliminated in R16

Distortions

Want a simple metric for match-level distortions

  • Consider two teams \(i\) and \(j\) that are not excluded from matching to a third team \(k\) via:
    • \( \Delta P_{i,j,k} :=\left|P_{i,k} - P_{j,k}\right| \)
  • Considering average distortion across all feasible \((i,j,k)\) comparisons \(Q(P) \)

 

If all equally treated teams have equal odds of matching, then this is minimized at \(Q=0\)

Distortions

Are Fairer designs Possible?

  • The UEFA design is centered around transparency to outside observers.
  • Question: Does transparency come at a fairness cost, increasing the distortions?
  • Answer possible due to Market Design tool: 

Are Fairer designs Possible?

A biheirarchy here means the constraints can be divided into two groups of constraints with subset ordering

Match exclusions

Row team has exactly 1 partner

Column team has exactly 1 partner

Heirachy \(\mathcal{H}_1\)

Heirachy \(\mathcal{H}_2\)

\(\Rightarrow\) Any expected assignment matrix satisfying the constraints is implementable as a lottery over \(\Gamma\)

Any expected assignment matrix satisfying the constraints is implementable as a lottery over \(\Gamma\)

But this means we can choose an expected assignment matrix \(\mathbf{P}\) (that satisfies the constraints) to maximize:

 

Q(P)=\frac{1}{\left| \Upsilon \right|} \sum_{(i,j,k)\in \Upsilon}\left|P _{i,k}-P _{j,k}\right|

Reduce the dimensionality of the problem from ~6000 degrees of freedom to ~50 !

  • Optimal fairness design can increase equality of outcomes by ~10%
  • But this must be set against reduced transparency
    • optimal constructions will be a black box 

To all extents and purposes, the conclusion is that given the constraints, the UEFA procedure is essentially fair!

Weakening the Constraints

While the mechanism is close to optimal given the constraints, we can employ similar analyses to understand the fairness gains from weakening the constraints.

\rightarrow \left\{\mu \in \mathcal{A}\Bigl| \left| \mu \cap H\right|\leq 1 \bigr.\right\}
\Gamma_H := \left\{\mu \in \mathcal{A}\Bigl| \left| \mu \cap H\right| = 0 \bigr.\right\}

So the weakened constraints allows for at most one of the excluded matches, where the other is complete proscription

Relaxed constraint Set

Generalization Example

Filling three committees:

  • CMTE X: 2 Seniors and 1 Junior
  • CMTE Y: At least 1 senior, 3 total
  • CMTE Z: 3 Seniors

People:

  • Seniors at most 2
  • Juniors at most 1
  • \(S_2\) and \(S_3\) can't serve together
  • Chair can't serve on \(X\)
  • One-to-one matching exhibits near optimality
  • Not the case in many-to-many (here demonstrated by counter-example) as expected assignment here has a great deal of inequality in usage
  • The following expected assignment is feasible (and implementable as are biheirarchy)
  • However, can be implemented by the simple dynamic draw:
    • Need to add further constraints!

Conclusions

  • Document a transparent randomization, designed to solve a complex combinatoric assignment
    • Organization developed a solution under a behavioral constraints that would not initially be obvious to economists
  • However, economists do have a great set of tools that can aid with understanding and assessing the design
  • Moreover, through these tools, we can find the generalizations that can extend to other settings

Make Sure to Bring a Coat