The behavioural effects of individual-level message tailoring

ESPA 2025: P4-S100 POLCOM

Lennard Metson

London School of Economics

26 June 2025

Background

Campaign organisations mobilise participation and structure democratic competition

  • New technology → use collected data to automatically tailor communication
  • Message tailoring could help organise and mobilise increasingly diffuse supporter bases at scale (Kreiss, 2016)

However, these technologies could have negative externalities. Message tailoring could:

Is individual-level message tailoring effective?

Message tailoring

(1) the use of information about a message recipient to (2) adapt message content and, (3) send different message variations to individuals

Message tailoring varies by the:

  • source of the information about recipients (e.g. disclosed, obtained, modelled)
  • type & number of variables used to adapt messages
  • method of targeting

Literature

Existing literature

Evidence from survey experiments suggests that tailoring can be effective (e.g. Gahn, 2024; Robinson, 2023).

But there is a strong risk of backlash, especially when:

Existing literature

(1) Existing literature has looked mostly at tailoring to groups

  • Campaigns are increasingly collecting and using individual-level data (Dommett et al., 2024)
  • Tailoring based on disclosed data: two-way communication between campaigns → opens up new mechanisms

Tailoring which uses previously-disclosed information may work differently.

Existing literature

(2) Existing literature is reliant on survey experiments → limits ecological validity (Kinder, 2007)

  • Lack of behavioural outcomes in surveys
  • Compared to survey settings, campaigns face:
    • Lower quality data
    • Restricted range of plausible messages

Taken together, tailoring in the field is less likely to be effective than in a survey setting.

Expectations

Expectations

This paper tests:

  • mobilisation messages from a campaign organisation to its supporters, asking them to take action
  • messages tailored to the recipient’s previously disclosed most important issue
  • two versions of tailoring: stated, and unstated tailoring

Expectations

Tailoring might work by:

Stated tailoring signals responsiveness to information, and may additionally:

  • be seen as an endorsement of goals → “relational conditions” (Han, 2016)
  • increase efficacy about ability to shape the organisation’s agenda → perceived selective political benefits (Scarrow, 2015)
  • remind recipients of their previous commitments → pressure

Hypotheses

H1: Tailoring a message to an individual’s most important issue will increase the probability that they take action.

H2: Tailoring a message to an individual’s most important issue and referencing why the issue was chosen will be more effective than tailoring alone.

Design

Collaborative field experiment with a pressure group during a petition campaign calling for proportional representation.

  • Sample (\(N=11,743\)): all subscribers who
    • had not already signed the petition1
    • had previously disclosed a most important issue

Design

  • Signatures to the petition measured after 2 and 10 days

  • Randomly assigned to pure control (no email), or 1 of 3 email conditions…

Design: Treatments

T1: Untailored

When we asked you what matters most to you, you said [own_issue] was one of the most important issues facing the UK.

By giving everyone an equal voice and vote, electoral reform ensures urgent issues like [rand_other_issue] will remain at the top of parliament’s agenda.

Design: Treatments

T2: Unstated tailored

When we asked you what matters most to you, you said [own_issue] was one of the most important issues facing the UK.

By giving everyone an equal voice and vote, electoral reform ensures urgent issues like [own_issue] will remain at the top of parliament’s agenda.

Design: Treatments

T3: Stated tailored

When we asked you what matters most to you, you said [own_issue] was one of the most important issues facing the UK.

By giving everyone an equal voice and vote, electoral reform ensures urgent issues like [own_issue] will remain at the top of parliament’s agenda.

Results

Any email vs pure control

Results

Any tailored vs untailored

Results

Unstated tailored vs untailored

Results

Stated tailored vs untailored

Results

Stated vs Unstated tailored

Conclusions

Tailoring using individual issue priorities can be effective, but only when explicitly stated.

  • What are the mechanisms behind stated tailoring?
  • Is stated tailoring effective when campaigns communicate beyond their supporters?

In the field, unstated tailoring was ineffective → importance of testing tactics as they are used.

  • Outdated issue priorities?
  • Limited range of issues?

Appendix

Issues selected

Issue N
Climate crisis 3748
Rejoining the EU 3261
Health and Social Care 2312
Wealth Inequality 1337
Cost of living 917
Controlling AI 79
Race Inequality 52
Gender Inequality 37

Treatment response timing

Random assignment diagram

Main results as tables: 10 days

(1)
(2)
Comparison ITT p (RI) ITT p (RI)
Any email (TG1, TG2, TG3) Vs PC 0.013* 0.0111 0.013* 0.0114
Stated & unstated tailored (TG2,TG3) Vs PC 0.015** 0.0044 0.015** 0.0044
Stated tailored (TG3) Vs PC 0.018** 0.0010 0.018** 0.0011
Non-stated tailored (TG2) Vs PC 0.011* 0.0369 0.011* 0.0406
Untailored (TG1) Vs PC 0.009 0.1049 0.009 0.1108
Non-stated tailored (TG2) Vs untailored (TG1) 0.002 0.5520 0.002 0.5674
Stated tailored (TG3) Vs untailored (TG1) 0.009* 0.0283 0.009* 0.0238
Stated tailored (TG3) Vs unstated tailored (TG2) 0.006 0.1055 0.007+ 0.0888
Any tailored email (TG2, TG3) Vs Untailored email (TG1) 0.005 0.1051 0.006+ 0.0990

Factorial linear models for effects on petition signatures after 2 and 10 days. All models control for pre-treatment signatures to deal with imbalance. The adjusted models additionally control for the count of previous actions the supporter has participated in. ITTs estimated using difference-in-proportions for outcome measured 10 days after treatment. To deal with imbalance, both models control for whether the participant had signed the petition before treatment emails were sent. Model (1) contains no further controls. Model (2) controls for the count of previous campaigns the supporter had participated in. \(p\)-values were calculated using randomisation inference. *** = p < 0.001, ** = p < 0.01, * = p < 0.05, + = \(p\) < 0.1.

Main results as tables: 2 days

(1)
(2)
Comparison ITT p (RI) ITT p (RI)
Any email (TG1, TG2, TG3) Vs PC 0.014** 0.0023 0.014** 0.0024
Stated & unstated tailored (TG2,TG3) Vs PC 0.016*** 0.0006 0.016*** 0.0009
Stated tailored (TG3) Vs PC 0.018*** 0.0003 0.018*** 0.0003
Non-stated tailored (TG2) Vs PC 0.013** 0.0089 0.013** 0.0093
Untailored (TG1) Vs PC 0.01+ 0.0545 0.01+ 0.0573
Non-stated tailored (TG2) Vs untailored (TG1) 0.003 0.3824 0.003 0.3905
Stated tailored (TG3) Vs untailored (TG1) 0.008* 0.0185 0.009* 0.0150
Stated tailored (TG3) Vs unstated tailored (TG2) 0.005 0.1338 0.006 0.1125
Any tailored email (TG2, TG3) Vs Untailored email (TG1) 0.006+ 0.0601 0.006+ 0.0562

Factorial linear models for effects on petition signatures after 2 and 10 days. All models control for pre-treatment signatures to deal with imbalance. The adjusted models additionally control for the count of previous actions the supporter has participated in. ITTs estimated using difference-in-proportions for outcome measured 10 days after treatment. To deal with imbalance, both models control for whether the participant had signed the petition before treatment emails were sent. Model (1) contains no further controls. Model (2) controls for the count of previous campaigns the supporter had participated in. \(p\)-values were calculated using randomisation inference. *** = p < 0.001, ** = p < 0.01, * = p < 0.05, + = \(p\) < 0.1.

Conditional effects by pre-treatment issue

Conditional effects by pre-treatment issue

Conditional effects by pre-treatment issue

Conditional effects by pre-treatment issue

Balance: pre-registered

Imbalance on pre-treatment signatures

Signed before email indicator
  • p < 0.1, * p < 0.05, ** p < 0.01, *** p < 0.001
(Intercept) 0.184***
(0.011)
Untailored 0.028*
(0.013)
Unstated tailored 0.037**
(0.013)
Stated tailored 0.025+
(0.013)
N 11,743
R² 0.001

Bandwidths

References

Chong, & Druckman. (2007). Framing theory. Annual Review of Political Science, 10(1), 103–126. https://doi.org/10.1146/annurev.polisci.10.072805.103054
Dommett, K., Kefford, G., & Kruschinski, S. (2024). Data-driven campaigning and political parties: Five advanced democracies compared. Oxford University Press. https://doi.org/10.1093/oso/9780197570227.001.0001
Gahn, C. (2024). How much tailoring is too much? Voter backlash on highly tailored campaign messages. The International Journal of Press/Politics, 19401612241263192. https://doi.org/10.1177/19401612241263192
Gibson, R., Bon, E., & Dommett, K. (2024). "I always feel like somebody’s watching me": What do the u.s. Electorate know about political micro-targeting and how much do they care? Journal of Quantitative Description: Digital Media, 4. https://doi.org/10.51685/jqd.2024.001
Han, H. (2016). The organizational roots of political activism: Field experiments on creating a relational context. American Political Science Review, 110(2), 296–307. https://doi.org/10.1017/S000305541600006X
Hersh, E., & Schaffner, B. F. (2013). Targeted campaign appeals and the value of ambiguity. The Journal of Politics, 75(2), 520–534. https://doi.org/10.1017/s0022381613000182
Kinder, D. R. (2007). Curmudgeonly advice. Journal of Communication, 57(1), 155–162. https://doi.org/10.1111/j.1460-2466.2006.00335.x
Kreiss, D. (2016). Prototype politics: Technology-intensive campaigning and the data of democracy. Oxford University Press.
Lawall, K., Turnbull-Dugarte, S., Foos, F., & Townsley, J. (2024). Negative political identities and costly political action. The Journal of Politics, 730718. https://doi.org/10.1086/730718
Robinson, T. S. (2023). When do voters respond to campaign finance disclosure? Evidence from multiple election types. Political Behavior, 45(4), 1309–1332. https://doi.org/10.1007/s11109-021-09766-y
Scarrow, S. E. (2015). Beyond party members: Changing approaches to partisan mobilization. Oxford University Press. https://doi.org/10.1093/acprof:oso/9780199661862.001.0001