The behavioural effects of individual-level message tailoring

EPOP 2025 | Panel 3.1: Campaigns

Lennard Metson

London School of Economics

5 September 2025

Background

Technological developments allow campaigns to automate tailored communication → “micro-targeting”

We have little evidence about whether, and how, it works in real world settings

Background

Assumed mechanism: heterogeneity → total effect of a campaign ↑ if each subgroup receives the message they are most responsive to

Argument of this paper:

Is individual-level message tailoring effective in the field?

Message tailoring

(1) the use of information about a message recipient to (2) adapt message content and, (3) send different message variations to individuals

Message tailoring varies by the:

  • source of the information about recipients (e.g. disclosed, obtained, modelled)
  • type & number of variables used to adapt messages
  • method of targeting

Literature

Existing literature

Evidence from survey experiments suggests that tailoring can be effective (e.g. Gahn, 2024; Robison et al., 2021; Tappin et al., 2023).

But there is a strong risk of backlash, especially when:

Existing literature

(1) Existing literature is reliant on survey experiments → limits ecological validity (Carnes & Henderson, 2025; Kinder, 2007)

  • Lack of behavioural outcomes in surveys
  • Compared to survey settings, campaigns face:
    • Lower quality data
    • Restricted range of plausible messages

Taken together, tailoring in the field is less likely to be effective than in a survey setting.

Existing literature

(2) Existing literature has looked mostly at tailoring to groups

  • Campaigns are increasingly collecting and using individual-level data (Dommett et al., 2024)
  • Tailoring based on disclosed data: two-way communication between campaigns → opens up new mechanisms

Tailoring which uses previously-disclosed information may work differently.

Expectations

Expectations

This paper tests:

  • mobilisation messages from a campaign organisation to its supporters, asking them to take action
  • messages tailored to the recipient’s previously disclosed most important issue
  • two versions of tailoring: stated, and unstated tailoring

Expectations

Tailoring might work by:

Stated tailoring signals responsiveness to information, and may additionally:

  • be seen as an endorsement of goals → “relational conditions” (Han, 2016)
  • increase efficacy about ability to shape the organisation’s agenda → perceived selective political benefits (Scarrow, 2015)
  • remind recipients of their previous commitments → pressure

Hypotheses

H1: Tailoring a message to an individual’s most important issue will increase the probability that they take action.

H2: Tailoring a message to an individual’s most important issue and referencing why the issue was chosen will be more effective than tailoring alone.

Design

Collaborative field experiment with a pressure group during a petition campaign calling for proportional representation.

  • Sample (\(N=11,743\)): all subscribers who
    • had not already signed the petition1
    • had previously disclosed a most important issue

Design

  • Signatures to the petition measured after 2 and 10 days

  • Randomly assigned to pure control (no email), or 1 of 3 email conditions…

Design: Treatments

T1: Untailored

When we asked you what matters most to you, you said [own_issue] was one of the most important issues facing the UK.

By giving everyone an equal voice and vote, electoral reform ensures urgent issues like [rand_other_issue] will remain at the top of parliament’s agenda.

Design: Treatments

T2: Unstated tailored

When we asked you what matters most to you, you said [own_issue] was one of the most important issues facing the UK.

By giving everyone an equal voice and vote, electoral reform ensures urgent issues like [own_issue] will remain at the top of parliament’s agenda.

Design: Treatments

T3: Stated tailored

When we asked you what matters most to you, you said [own_issue] was one of the most important issues facing the UK.

By giving everyone an equal voice and vote, electoral reform ensures urgent issues like [own_issue] will remain at the top of parliament’s agenda.

Results

Any email vs pure control

Results

Any tailored vs untailored

Results

Unstated tailored vs untailored

Results

Stated tailored vs untailored

Results

Stated vs Unstated tailored

Conclusions

Tailoring using individual issue priorities can be effective, but only when explicitly stated.

  • What are the mechanisms behind stated tailoring?
  • Is stated tailoring effective when campaigns communicate beyond their supporters?

In the field, unstated tailoring was ineffective → importance of testing tactics as they are used.

  • Outdated issue priorities?
  • Limited range of issues?

Thank you!

📧 l.m.metson@lse.ac.uk

🌐 lenmetson.com

Appendix

Issues selected

Issue N
Climate crisis 3748
Rejoining the EU 3261
Health and Social Care 2312
Wealth Inequality 1337
Cost of living 917
Controlling AI 79
Race Inequality 52
Gender Inequality 37

Treatment response timing

Random assignment diagram

Main results as tables: 10 days

(1)
(2)
Comparison ITT p (RI) ITT p (RI)
Any email (TG1, TG2, TG3) Vs PC 0.013* 0.0111 0.013* 0.0114
Stated & unstated tailored (TG2,TG3) Vs PC 0.015** 0.0044 0.015** 0.0044
Stated tailored (TG3) Vs PC 0.018** 0.0010 0.018** 0.0011
Non-stated tailored (TG2) Vs PC 0.011* 0.0369 0.011* 0.0406
Untailored (TG1) Vs PC 0.009 0.1049 0.009 0.1108
Non-stated tailored (TG2) Vs untailored (TG1) 0.002 0.5520 0.002 0.5674
Stated tailored (TG3) Vs untailored (TG1) 0.009* 0.0283 0.009* 0.0238
Stated tailored (TG3) Vs unstated tailored (TG2) 0.006 0.1055 0.007+ 0.0888
Any tailored email (TG2, TG3) Vs Untailored email (TG1) 0.005 0.1051 0.006+ 0.0990

Factorial linear models for effects on petition signatures after 2 and 10 days. All models control for pre-treatment signatures to deal with imbalance. The adjusted models additionally control for the count of previous actions the supporter has participated in. ITTs estimated using difference-in-proportions for outcome measured 10 days after treatment. To deal with imbalance, both models control for whether the participant had signed the petition before treatment emails were sent. Model (1) contains no further controls. Model (2) controls for the count of previous campaigns the supporter had participated in. \(p\)-values were calculated using randomisation inference. *** = p < 0.001, ** = p < 0.01, * = p < 0.05, + = \(p\) < 0.1.

Main results as tables: 2 days

(1)
(2)
Comparison ITT p (RI) ITT p (RI)
Any email (TG1, TG2, TG3) Vs PC 0.014** 0.0023 0.014** 0.0024
Stated & unstated tailored (TG2,TG3) Vs PC 0.016*** 0.0006 0.016*** 0.0009
Stated tailored (TG3) Vs PC 0.018*** 0.0003 0.018*** 0.0003
Non-stated tailored (TG2) Vs PC 0.013** 0.0089 0.013** 0.0093
Untailored (TG1) Vs PC 0.01+ 0.0545 0.01+ 0.0573
Non-stated tailored (TG2) Vs untailored (TG1) 0.003 0.3824 0.003 0.3905
Stated tailored (TG3) Vs untailored (TG1) 0.008* 0.0185 0.009* 0.0150
Stated tailored (TG3) Vs unstated tailored (TG2) 0.005 0.1338 0.006 0.1125
Any tailored email (TG2, TG3) Vs Untailored email (TG1) 0.006+ 0.0601 0.006+ 0.0562

Factorial linear models for effects on petition signatures after 2 and 10 days. All models control for pre-treatment signatures to deal with imbalance. The adjusted models additionally control for the count of previous actions the supporter has participated in. ITTs estimated using difference-in-proportions for outcome measured 10 days after treatment. To deal with imbalance, both models control for whether the participant had signed the petition before treatment emails were sent. Model (1) contains no further controls. Model (2) controls for the count of previous campaigns the supporter had participated in. \(p\)-values were calculated using randomisation inference. *** = p < 0.001, ** = p < 0.01, * = p < 0.05, + = \(p\) < 0.1.

Conditional effects by pre-treatment issue

Conditional effects by pre-treatment issue

Conditional effects by pre-treatment issue

Conditional effects by pre-treatment issue

Balance: pre-registered

Imbalance on pre-treatment signatures

Signed before email indicator
  • p < 0.1, * p < 0.05, ** p < 0.01, *** p < 0.001
(Intercept) 0.184***
(0.011)
Untailored 0.028*
(0.013)
Unstated tailored 0.037**
(0.013)
Stated tailored 0.025+
(0.013)
N 11,743
R² 0.001

Bandwidths

References

lenmetson.com

Broockman, D. E., Kalla, J. L., Caballero, C., & Easton, M. (2024). Political practitioners poorly predict which messages persuade the public. Proceedings of the National Academy of Sciences, 121(45), e2400076121. https://doi.org/10.1073/pnas.2400076121
Carnes, N., & Henderson, G. L. (2025). Not getting the message on climate? Attention as a key barrier to mass-marketing experimentally-validated messages. British Journal of Political Science, 55, e106. https://doi.org/10.1017/S000712342510063X
Chong, & Druckman. (2007). Framing theory. Annual Review of Political Science, 10(1), 103–126. https://doi.org/10.1146/annurev.polisci.10.072805.103054
Coppock, A. (2022). Persuasion in parallel: How information changes minds about politics. The University of Chicago Press. https://doi.org/10.7208/chicago/9780226821832.001.0001
Dommett, K., Kefford, G., & Kruschinski, S. (2024). Data-driven campaigning and political parties: Five advanced democracies compared. Oxford University Press. https://doi.org/10.1093/oso/9780197570227.001.0001
Gahn, C. (2024). How much tailoring is too much? Voter backlash on highly tailored campaign messages. The International Journal of Press/Politics, 19401612241263192. https://doi.org/10.1177/19401612241263192
Gibson, R., Bon, E., & Dommett, K. (2024). "I always feel like somebody’s watching me": What do the u.s. Electorate know about political micro-targeting and how much do they care? Journal of Quantitative Description: Digital Media, 4. https://doi.org/10.51685/jqd.2024.001
Han, H. (2016). The organizational roots of political activism: Field experiments on creating a relational context. American Political Science Review, 110(2), 296–307. https://doi.org/10.1017/S000305541600006X
Hersh, E. (2015). Hacking the electorate: How campaigns perceive voters. Cambridge University Press.
Hersh, E., & Schaffner, B. F. (2013). Targeted campaign appeals and the value of ambiguity. The Journal of Politics, 75(2), 520–534. https://doi.org/10.1017/s0022381613000182
Hewitt, L., Broockman, D., Coppock, A., Tappin, B. M., Slezak, J., Coffman, V., Lubin, N., & Hamidian, M. (2024). How experiments help campaigns persuade voters: Evidence from a large archive of campaigns’ own experiments. American Political Science Review, 1–19. https://doi.org/10.1017/S0003055423001387
Kinder, D. R. (2007). Curmudgeonly advice. Journal of Communication, 57(1), 155–162. https://doi.org/10.1111/j.1460-2466.2006.00335.x
Kreiss, D. (2016). Prototype politics: Technology-intensive campaigning and the data of democracy. Oxford University Press.
Lawall, K., Turnbull-Dugarte, S., Foos, F., & Townsley, J. (2024). Negative political identities and costly political action. The Journal of Politics, 730718. https://doi.org/10.1086/730718
Robison, J., Stubager, R., Thau, M., & Tilley, J. (2021). Does class-based campaigning work? How working class appeals attract and polarize voters. Comparative Political Studies, 54(5), 723–752. https://doi.org/10.1177/0010414020957684
Scarrow, S. E. (2015). Beyond party members: Changing approaches to partisan mobilization. Oxford University Press. https://doi.org/10.1093/acprof:oso/9780199661862.001.0001
Tappin, B. M., Wittenberg, C., Hewitt, L. B., Berinsky, A. J., & Rand, D. G. (2023). Quantifying the potential persuasive returns to political microtargeting. Proceedings of the National Academy of Sciences, 120(25), e2216261120. https://doi.org/10.1073/pnas.2216261120