Using Natural Experiments to Uncover Effects of Anti-Refugee Riots on Attitudes of Refugees

While previous research has focused on terrorist attacks and natives’ attitudes towards immigration, we examine the effect of anti-refugee attacks on refugees’ attitude towards the host country. We use survey data from the 33rd wave of the German Socio-Economic Panel as the fieldwork period overlapped with the infamous anti-refugee riots in Bautzen and as the survey includes a refugee sample. Making use of this natural experiment, we find significant and negative short-term effects of the riots on respondents’ perception of Germany, as well as low geographic variation. Such natural experiments in the form of unexpected events during survey design offer social scientists the possibilities to identify causal effects from observational survey data as they split respondents into a control and treatment group. Given the vast amount of (cross-)national survey data, often including specific subsamples, our study demonstrates the great potential of natural experiments for sociological research on minority groups in society.


Introduction
In the last decade, many European (and non-European) countries experienced a large influx of refugees and large parts of the host societies were welcoming and supported their integration into society (Akrap, 2016).However, there was also a dark side in the form of increasing numbers of hate crimes towards refugees and ethnic minorities (Pai, 2020).Little is known about the causal effects of such hate crimes on refugees: While both Ben Farhat et al. (2018) and Bouhenia et al. (2017) document and describe experiences with violence and psychological distress of refugees using interview data collected in Greece and in France, respectively, and Schwitter and Liebe (2022) use population survey data to provide correlational evidence that refugees feel less positive about Germany in regions with a high accumulated share of arson attacks on refugee homes, the identification of causal effects remains challenging with such data and analytical approaches.A better understanding on how refugees perceive their host country and how this perception can change over time provides important insights into the factors that influence refugees' social integration and their overall well-being.
While randomised experiments are considered the gold standard in causal analysis (Rubin, 2008), they are often extremely difficult or even unethical to implement, especially when researching sensitive topics.Particularly in such instances, naturally occurring experiments can offer a valuable and useful study design.Unlike controlled experiments, natural experiments occur when circumstances or events create a situation where individuals are exposed to experimental and to control conditions depending on factors outside of the researchers' control.Thus, in natural experiments, the assignment of individuals to different conditions is determined by external factors.They often arise from naturally occurring events, policy changes, geographical variation or historical incidents (see Dunning, 2012).Unforeseen and salient events during the fieldwork of public opinion surveys can also work as such external factors and split the sample of respondents into a treatment -those surveyed after the event -and a control group -those surveyed before the event (Muñoz et al., 2020); they are also called unexpected events during survey design (UESD).When specific assumptions are met, causal effects can be assessed with this setup. 1The advantage of natural experiments lies in their ability to approximate the conditions of a controlled experiment while taking advantage of existing real-world situations.By leveraging these situations, researchers can observe the effects of an independent variable (the external factor) on variables of interest by comparing the control and the treatment group, providing insights into causal relationships.
In this research note, we consider the riots in the East-German town of Bautzen to be an unexpected event during the survey fieldwork of the 33rd wave of the German Socio-Economic Panel (SOEP).Employing a quasi-experimental approach through this setup will allow us to assess potential causal effects of the incident on refugees' perception on welcomeness.This extends previous work on the attitudes of refugees towards their host country as it is the first study that shows effects of violent attacks on refugees' attitudes in an experimental framework.In the next section, we will introduce our study context before we then present the effects found.We will conclude with a discussion of UESD and point to further example studies to highlight the method's applicability.

A Riot on Refugees as an Unexpected Event During Survey Design
The Eastern German town of Bautzen became infamous for an extreme event of ethnic violence in 2016: after a series of events, including arson attacks on a hotel building initially designated for use as refugee accomodation in February, as well as several national demonstrations throughout the summer, tensions reached a peak on the night of September 14/15 2016.This culminated in a 'hunt for refugees' when about 80 violent local right-wing extremists clashed with 15-20 young unaccompanied refugees (Deutsche Welle (DW), 2016).This incident, during which several people were verbally harassed and physically wounded, has received notable media attention.Ethnic violence has not come to a halt in Bautzen after this incident; in December of the same year, another refugee accommodation was attacked.In the following, we will assess the effect of the Bautzen riots of 15 September 2016 on the refugee population in Germany.
Ethnic violence can be seen as a perceived 'realistic threat' (Stephan and Stephan, 2000: 25;also Stephan et al., 2002) to the well-being and physical safety of refugees, and we expect that extreme incidents of violence towards refugees such as riots, which were reported in local and national media, will lead refugees to feel more negatively towards their host country.Past research has made use of groupthreat theory to explain levels of prejudice against ethnic minorities, arguing that they have been perceived as threats (see, for example, Legewie, 2013).In a metaanalytical review, Riek et al. (2006) discussed how different threat types, including realistic threats, lead to more negative outgroup attitudes.Against this background, we argue that refugees consider the population of the host country as a threat after experiencing or hearing about an incident and that they thus develop more negative attitudes about the host country.However, the temporal duration of such an attitudinal change might vary with the study context and needs to be explored (e.g.Frey, 2022;Legewie, 2013).
Our analysis is based on the German SOEP (Goebel et al., 2019; Socio-Economic Panel (SOEP), 2019), a longitudinal household survey of the German population.Since 2016, the SOEP also surveys refugees, sampled from the Central Register of Foreign Nationals (Brücker et al., 2016(Brücker et al., , 2019)).The sample includes individuals who have entered Germany since January 2013 with an asylum application.The fieldwork period of the 2016 SOEP survey overlapped with the riots in Bautzen, which took place on the night of 14/15 September 2016 (see Figure 1).This circumstance allows us to assess whether the riots had an impact on the attitudes of refugees by comparing respondents interviewed prior to the attacks (control group) to those after (treatment group).For the 2016 survey, the sample includes 4465 individuals nested in 3289 households.After removing observations with missing answers in relevant variables (see below), the sample for the full fieldwork period consists of 4078 respondents in 3022 households (1742 respondents in the control group, 2336 in the treatment group).For the fieldwork period ±80 days around the riots (the longest bandwidth we consider in this study), the sample consists 1314 respondents nested in 1022 households (699 respondents in the control group, 615 in the treatment group).

Dependent Variable: Perceived Welcomeness
We use one attitudinal variable on perceived welcomeness to assess the impact of the Bautzen riots on refugees and examine whether the attack impacted their attitudes towards Germany.This variable provides a holistic measure, particularly given the background of the German 'welcome culture' and has been used in previous research (see Frey, 2022;Schwitter and Liebe, 2022).In the survey, refugees are asked how welcome they currently feel in Germany, and they can answer on a scale from 0 (not at all) to 4 (completely).The overwhelming majority of respondents feels welcome in Germany; in the full sample, 61.6% of respondents in the control group and 63.5% of respondents in the treatment group feel completely welcome in Germany; 23.8% and 22.3% for the most part, 11.7% and 10.5% in some respects, 2.1% and 2.6% barely, and 0.8% and 1.1% not at all.Given this skewed distribution and the small sample size for some response categories, we simplify the analysis by collapsing categories and dichotomising the variable (completely/for the most part vs. the rest).

Experimental Design
We expect the Bautzen riots to work as an exogenous stimulus; respondents are randomly assigned to the treatment and the control group, depending only on the timing of their interview.The main independent variable in our analysis thus forms the treatment indicator, which takes a value of 0 if a refugee was interviewed before the Bautzen riot (i.e.before 15 September) and 1 if the interview took place after.Given that the attacks took place during the night, respondents interviewed on 15 September are in the treatment group.To assess a causal effect, two assumptions must hold in the UESD: excludability and temporal ignorability (Muñoz et al., 2020).The excludability criterion states that the differences in attitudes are solely due to the treatment effect and that there is no other channel than the riot.This cannot be guaranteed as time-varying variables systematically related to the outcome may bias the estimate.To limit such biases and capture temporal fading of effects, we consider different bandwidths around the event (between ±80 days and ±4 days around the event).We conduct imbalance analyses for different time frames around the event date to check whether the UESD fully satisfies the temporal ignorability assumption, that is, whether control and treatment group differ significantly in terms of specific variables other than the outcome (see Supplemental material).The present UESD does not fully satisfy the ignorability assumption.To relax the assumption, we include information on the number of prior interview attempts (see a similar approach in, for example, Frey, 2022).To further control for differences in reachability and survey design, we control whether respondents live in East or West Germany, and for geographic proximity to Bautzen (measured as the logarithm of the Euclidean distance to respondents' districts centre), age, gender, marital status, citizenship, German language skills, type of accommodation, asylum application status and years already living in Germany.In addition to conditioning on these variables in the model estimation, we further match the sample using entropy balancing (Hainmueller, 2012;Hainmueller and Xu, 2013).We also check whether there is a pre-existing negative time trend in our dependent variable before the Bautzen riot; this is not the case (see Online Supplemental Material).
Under these assumptions, the causal effect of the Bautzen riot on attitudinal change can be estimated as follows: with T i being a binary variable differentiating the control and the treatment group, that is, being an indicator of time, so that β 1 identifies the estimated effect of the riot, conditional on covariates X i .We employ linear probability models with cluster-robust standard errors to account for heteroskedasticity and clustering at the household level. 2

Effects of Riots on Refugees' Attitudes towards the Host Country
How did the Bautzen riots affect the perceived welcomeness of refugees in Germany?In short bandwidths around the event -comparing respondents interviewed, for example, in the eight days before the attack with those interviewed in the eight days after the attack -we find a significant and negative treatment effect of 7.5%.This means, the refugees surveyed feel significantly less welcome in Germany after the violent Bautzen riots.In Figure 2, we plot the treatment effect of Bautzen (model including controls) when increasing the bandwidth from four days around the Bautzen riot to 80 days around the riots in steps of four-day bandwidths (see, for a similar plotting approach, Frey, 2022).
Figure 2 captures the fading effect over time: while the treatment effect of Bautzen is strong and significant in the recent aftermath (with the exception of the 12-day bandwidth where the effect is not as pronounced), it weakens in strength and significance over time (see Table A.4 in the Supplemental material for the numerical results).Please note that the effect does not reach conventional significance thresholds when using a bandwidth of four days, due to the small sample size.
For better interpretation, we calculate average predictive margins based on these models.We find that the average probability to feel welcome in Germany lies at 90.7% four days before the Bautzen attack and drops to 83.3% in the four days after the attack.This difference decreases the larger the bandwidth: the average probability to feel welcome in the 16 days before the riots lies at 88.6% and drops to 82.9% after the attack; in the 28 days before the riots, it lies at 86.6% and drops to 82.7%; and when comparing those interviewed 80 days before and after the Bautzen attack, there is a less than 2% difference with an average predicted probability of 85.8% to feel welcome in Germany if surveyed before and 84.5% if surveyed after.We also checked whether the negative treatment effect is significantly weaker the further away from Bautzen respondents live, however, we only find a significant interaction effect between the treatment and the distance from Bautzen in the very recent aftermath (up to 12 days after the event; see Online Supplemental Material). 3

Discussion and Conclusion: Making Use of Natural Experiments
Making use of an unexpected event and extending previous work, this study provides a more direct test of the causal effect of an incident of ethnic violence on refugees' attitudes towards Germany.Our negative treatment effect shows that violence against refugees influences their attitudes in the aftermath, highlighting the immediate and profound impact of negative events on refugees' perception of their host country.While we do not know yet what the individual consequences of such negative changes in attitude might be, it has shown to been accompanied with clinically relevant declines in mental health (Frey, 2022).The negative effect on the perceived welcomeness in Germany fades in strength and significance across time as the different bandwidths analysed suggest.While some other UESD studies have found long-term effects of extreme events (e.g.lasting 'rally effects' on support for George W Bush after 11 September 2001, see Hetherington and Nelson, 2003), it is not unusual that changes in attitudes are only temporary (see, for example, Frey, 2022).These findings suggest that negative events do not necessarily have lasting negative impacts on those affected; resilience and adaptation mechanisms might enable people to restore their opinions.Future research on this topic should investigate in more detail how refugees navigate the challenges of integrating into a host country when being met with hostility instead of hospitality.
Natural experiments offer a valuable tool to explore causal effects, and UESD studies have emerged as a promising area of research in the field of discrimination more generally (see, for example, Boomgaarden and De Vreese, 2007;Czymara and Schmidt-Catran, 2017;Echebarria-Echabe and Fernández-Guede, 2006;Frey, 2022;Jakobsson and Blom, 2014;Larsen et al., 2020;Legewie, 2013;Muñoz et al., 2020;Nägel and Lutter, 2020;Nussio et al., 2019;Schmidt-Catran and Czymara, 2020).In these experiments, distinct, unexpected events (most often terrorist attacks) occurring within the fieldwork of a survey programme were used as a quasi-experimental stimulus.Examples include: 2022) who, based on data from the SOEP and the ALLBUS (German General Social Survey; allgemeine Bevölkerungsumfrage der Sozialwissenschaften), found German respondents to feel increasingly hostile against refugees, as well as refugees to experience more discrimination, feel less welcome and suffer declines in mental health in the aftermath of a series of terrorist attacks (instead of attacks on refugees) in 2016.• • Larsen et al. (2020), Nägel and Lutter (2020) and Schmidt-Catran and Czymara (2020) employ the terrorist attack on the Berlin Christmas market in 2016 as a salient and unexpected event during the fieldwork of the European Social Survey (ESS).While Larsen et al. (2020) do not find an effect of the Berlin attack on attitudes towards refugees (they do not state their bandwidth), Schmidt-Catran and Czymara (2020) report that the change in attitudes was lagged.Nägel and Lutter (2020) find that immediately after the event, there is a decline in the acceptance of refugees by respondents with a right-wing political attitude only; as time progresses, this worsening of attitudes towards refugees can also be observed in the general population.• • Legewie (2013) finds considerable cross-national and regional variation in the effect and its temporal duration of the terrorist attack in Bali in October 2002 on attitudes towards immigration, based on ESS data from 65 regions in nine European countries.The effect of the Bali attack is found to be larger in regions with an increasing unemployment rate, and the immigrant share and potentials for contact with immigrants further shape the response to the event.
National and cross-national population surveys such as the British Social Attitudes survey, the United Kingdom Household Longitudinal Study (Understanding Society), the ESS and the SOEP used in this research note form essential sources of observational data for social scientists to better understand (changing) attitudes and behaviours of society.When the fieldwork of these studies overlaps with unforeseen and salient events, they can be used to estimate causal effects under certain assumptions.Many national (panel) surveys also include youth samples and systematic oversamples of subgroups; for example, the SOEP includes not only a specific sample covering refugees, but has also oversampled high-income households, households with different migration backgrounds and households with a household head who identifies as lesbian, gay, bisexual, transgender or non-binary.This allows researchers to analyse the effect of events on specific subgroups with high statistical power.While this UESD study design is not new, our research note demonstrates its potential in the context of research on ethnic violence and refugee integration.Given the vast amount of (cross-)national survey data available to researchers, the UESD type of natural experiments is a promising research strategy in several still unexplored research fields.

Figure 2 .
Figure 2. Treatment effect across different bandwidths, showing 5% significance level (thick black line) and 10% significance level (thin grey line).Plotted are coefficient estimates of a linear probability model.Plotting approach follows Frey (2022).