GV249 Seminar WT4: Field Experiments

Lennard Metson

2025-02-18

📊 ATE recap

Individual-level treatment effect: \(\tau_i = Y_i(1) - Y_i(0)\)
Average (individual-level) treatment effect: \(\text{mean}(Y_i(1) - Y_i(0)) = \text{mean}(Y_i(1)) - \text{mean}(Y_i(0))\)
\(D_i\) determines which \(Y_i\) is revealed →
- random assignment of \(D_i\) gives us random samples of \(Y_i(1)\) and \(Y_i(0)\) →
- unbiased estimate of \(\text{mean}(Y_i(1)) - \text{mean}(Y_i(0))\)

📊 ATE recap

But what if some people who we assigned treatment to don’t actually receive it?

\(D_i = 0\)?
No, because this would violate the independence assumption if whether or not subjects receive treatment is correlated with their POs.
When we have non-compliance, we can’t get at the sample ATE anymore. We need to choose the effect of what that we are interested in.

🙅 Non-compliance

There is now one more variable involved:

\(z_i\): binary variable indicating whether \(i\) was assigned to treatment
\(d_i\): binary variable indicating whether \(i\) was treated
\(Y_i\): the outcome for \(i\)

\(d_i(z)\): subject \(i\)’s potential treatment status
- \(d_i(z=1)\): treatment status when assigned to treatment
- \(d_i(z=0)\): treatment status when assigned to control

🙅 Non-compliance: types

Compliers: treatment always matches assignment
- \(d_i(z=1) = 1\) & \(d_i(z=0) = 0\)
Never-takers: never treated regardless of assignment
- \(d_i(z=1) = 0\) & \(d_i(z=0) = 0\)
Always-takers: always treated regardless of assignment
- \(d_i(z=1) = 1\) & \(d_i(z=0) = 1\)
Defiers: treatment status always the opposite of assignment
- \(d_i(z=1) = 0\) & \(d_i(z=0) = 1\)

🙅 Non-compliance: types

Why can’t we observe which compliance type our subjects are?

They only reveal one potential treatment status to us!

🙅 Non-compliance: ITT

The intent-to-treat (ITT) is the effect of treatment assignment on the outcome: \(Y_i(z=1) - Y_i(z=0)\)
- E.g. the effect of trying to treat someone. Often practitioners are more interested in this because it is the “effect of the programme”.
\(ITT = ATE\) when there is no non-compliance because treatment status is the same as treatment assignment if everyone complies.

🙅 Non-compliance: CACE

The complier-average-causal effect (CACE) is the average treatment effect amongst compliers.
To calculate the CACE, you divide the effect of assignment on the outcome (\(ITT_Y\)) by the effect of assignment on treatment (\(ITT_D\)):
- \(CACE = \frac{ITT_Y}{ITT_D}\)
\(CACE = ATE\) when there is no non-compliance because you are dividing by 1.

🙅 Non-compliance: CACE (assumptions)

Additional assumptions to identify the CACE:
- 1. no defiers
- 1. strong first-stage: there is an effect of assignment on treatment \(ITT_D\).
You can easily test assumption (2).
For (1), you need to think about how plausible defiance is given the specifics of the experiment. Usually it is quite unlikely.

🙅 Non-compliance: CACE (additional notes)

Additional context:

You might come across “2-stage least squares (2SLS) regression”. This is exactly the same idea but using OLS as your estimator instead of difference-in-means.
The CACE is equivalent to an Instrumental Variable design where treatment assignment is the instrument.

🏃‍♀️ Exercise

A political party wants to test whether sending their candidate to knock on doors makes people more likely to turn out to vote. They want you to design a field experiment to estimate the effect of door-to-door contact on turnout.

How will you assign subjects to treatment?
Consider the 3 key assumptions: (1) Independence; (2) Excludability; (3) Non-interference. What can you do to minimize violations?
What is non-compliance in this setting? How do we estimate the ITT and CACE? What do the different estimates mean?

You can use Townsley (2018) - Is it worth door-knocking? for inspiration.