GV249 Seminar WT5: Qualitative Field Research

Lennard Metson

2025-03-04

Recap: Estimators (ATE)

  • the average individual-level effect of \(D\) on \(Y\)
  • \(ATE = m(Y_i^1) - m(Y_i^0) = E[Y_i|D_i=1] - E[Y_i|D_i=0]\)
  • When there’s non-compliance \(Z_i \neq D_i\)

Recap: Estimators (ITT)

  • We decide to ignore \(D_i\), and focus on the effect of assignment (\(Z_i\)):
    • \(ITT = E[Y_i|Z_i=1] - E[Y_i|Z_i=0]\)
  • When everyone is a complier, \(Z_i = D_i\) → \(ITT = ATE\).

Recap: Estimators (CACE)

  • We try to get at the causal effect on the subgroup of compliers.
  • To do this, we need to scale the ITT by the proportion of compliers in our sample. The proportion of compliers is equal to the “effect” of \(Z_i\) on \(D_i\), which we call the \(ITT_D\).
    • \(ITT_D = E[D_i|Z_i=1] - E[D_i|Z_i=0]\)

\[ CACE = \frac{ITT_Y}{ITT_D} = \frac{E[Y_i|Z_i=1] - E[Y_i|Z_i=0]}{E[D_i|Z_i=1] - E[D_i|Z_i=0]} \]

Recap: Experiment assumptions

  • Independence: Treatment status is statistically independent of potential outcomes and background attributes
  • Excludability: When we only define two potential outcomes, \(Y_i^1\) and \(Y_i^1\), based on whether the treatment is administered, we assume that the only relevant causal agent is receipt of the treatment.
  • Non-interference: Each subject is unaffected by the treatments and assignments of other units.

Assumptions Quiz

Scan or click this link

Qualitative field research

Main types of qualitative research

“Field”

  • Participant observation
  • Surveys

“Desk”

  • Documentary analysis

Listening to People

Lareau - Listening to People (library link)

In the field

  • With both qualitative research and experiments in the field, the main challenge is balancing practical and research design considerations.
  • Need extra focus on ethics in the field

Participant observation

📖

Nielsen - Ground Wars, Methodological Appendix

  • Access: How do you “get in”?
  • Trust: How you you “stay in” and make people feel comfortable to be authentic?
  • Reflexivity: What is the impact of you observing? How are you changing the interactions you see?
  • Ethics

Interviews

Question types (from lecture slides):

  • Descriptive or Grand-tour: “Can you walk me through your day as a policymaker?”
  • Specific Grand-tour: “Could you walk me through what you did yesterday in you role as an MP?”
  • Opinion-based: “How do you feel about this policy?”
  • Behavioural: “Can you recall a moment when you had to negotiate a political compromise?”
  • Contrast: “How does grass-roots activism differ from formal party politics?”
  • Hypothetical: “If you were in charge of election reform, what’s the first change you’d implement?”
  • Probing: “You mentioned frustration with the media—can you elaborate on why?”

Interviews

Questions to avoid:

  • Leading questions
  • Loaded questions
  • Presuming questions (sometimes useful)

Sampling: How does the people you’re talking to affect the answers you get?

Mixed methods

Qualitative methods can be used in a primarily quantitative study to:

  • Validate assumptions
  • Explore potential mechanism of effects
  • Explain puzzling findings

Quantitative methods can be used in a primarily qualitative study to:

  • Explore how findings generalise
  • Test specific implications of a theory within a qualitative framework

🏃‍♀️ Exercise

Why is there a gender gap in people running for local office?

Design a qualitative study to answer this question:

  • What method(s) will you use? Participant observation? Interviews? Both?
  • What sample will you attempt to observe/interview?
  • What questions would you ask?
  • What challenges might you face? How can you address those?

🏃‍♀️ Exercise (optional)

Think back to last seminar: you designed a field experiment to estimate the effect of door-to-door contact on turnout.

You get the following results:

  1. Treated subjects are, on average, more likely to vote when they are contacted.
  2. However, when examining the effect on low-propensity voters in the sample, being contacted by the party makes them less likely to vote.

How could we use qualitative field research to make sense of these findings?

Extra notes on no-defiers assumption

Recap: No defiers

This is beyond the scope of this course. All you need to know is that we must assume no defiers when estimating the CACE.

  • \(ITT_D\) is how we estimate the proportion of the sample who are compliers.
    • Compliers exhibit a positive correlation
    • Never-takers exhibit no correlation
    • Always-takers exhibit no correlation
    • Defiers exhibit a negative correlation
  • \(\text{Cor}(Z,D) =\) compliance rate if there are no defiers, the slope of the line is driven only by the rate of compliers.

Recap: No defiers

GV249 WT5 | 📨 email l.m.metson@lse.ac.uk 🤔 Question? 🙋 raise your hand or 🖥️ use the Moodle Forum.