Data and Measures

In order to examine the relationship between civic education in high school and civic engagement in adulthood, we make use of the National Longitudinal Study of Adolescent to Adult Health (Add Health). Add Health is a school-based longitudinal study of a nationally representative sample of adolescents who were in grades 7–12 in the United States in 1994–95. Four waves of the Add Health study have been completed: Wave I (1994–1995), Wave II (1996), Wave III (2001–2002) and Wave IV (2008). Although the study was developed to explore the causes of health-related behavior of adolescents and their outcomes in young adulthood, it contains some variables of interest to political scientists, such as measures of voter turnout. The dataset also includes a range of variables that can be used as controls, such as cognitive ability, demographics and parental attributes.

Although the Add Health study is valuable on its own, the Academic Transcript Social Studies and Civic Coursework dataset was recently released as an auxiliary file. This dataset consists of academic transcript data related to social studies and civic coursework for those who participated in Wave III of the Add Health Study. By coding courses using the definition of social studies offered by the National Council on Social Studies and course content that is supported in the literature as important for future civic participation, this dataset provides an opportunity to examine the effect of civic education on outcomes in adulthood. The Adolescent Health and Academic Achievement (AHAA) study collected high school transcripts for approximately 12,000 Add Health participants who were part of the Wave III sample.Footnote ⁹ AHAA used the Classification of Secondary School Courses (CSSC) to code these transcripts.Footnote ¹⁰ AHAA followed the procedures used by the National Center for Education Statistics to code transcripts and trained coders using training materials from the 2000 National Assessment of Educational Progress High School Transcripts Study. The AHAA coders used extensive information on course content from schools to code transcripts. For each course, coders identified the primary content of the course listed on the transcript.Footnote ¹¹ The following categories were used: experiential learning, service learning, civic skills development, social and political issues/problems of society, historically marginalized groups, American history, international/multicultural studies and political knowledge development.Footnote ¹² If a social studies/civics course could not be grouped into one of these eight categories, it was classified as ‘other humanities/social science’. Because each course that a student took was coded, we were able to develop measures of the overall number of social studies/civics classes taken during high school and of the number of classes taken within each of the categories described above. Thus we have a general measure of the extent of participants’ exposure to civic education and a series of more nuanced measures that capture the types of courses (and how many of each type) they took in high school.

Merging the transcript data into the Add Health study provides an opportunity to examine the effect of civic education on voter turnout in adulthood. Indeed, the Add Health study asked respondents a number of questions about voting. In Wave III, participants were asked ‘Did you vote in the most recent presidential election?’ (we code yes = 1 and, no = 0). In Wave IV, participants were asked ‘How often do you usually vote in local or statewide elections?’ (always = 4, often = 3, sometimes = 2, never = 1). We use these two items as our dependent variables in the analyses that follow.Footnote ¹³

Before proceeding, it is important to note that self-reported measures of turnout are likely to be somewhat ‘noisy’ measures of participation. Indeed, scholars have long been concerned about the accuracy of measures of voting behavior, and there is now a body of research on over-reporting turnout. For instance, Karp and Brockington (Reference Karp and Brockington2005) found that some survey respondents may intentionally misreport that they voted because they want to be seen as complying with the social norm that voting in elections is desirable. In the context of this study, one concern is a potential link between family influences and self-reported turnout. For example, if self-reports are artificially high because of social desirability, it is possible that over-reporting is also due to family influences (that is, voting is valued more in some homes than others, which in turn leads to over-reporting and might explain why siblings socialized in the same home have comparable levels of self-reported turnout). However, if children are exposed to the same within-family norms related to voting, this would be captured by our family fixed-effects. Indeed, as we describe in more detail below, family fixed-effects models capture all shared observed and unobserved influences within families (for example, child-rearing practices, political socialization, shared genes, etc.). Overall, our goal in this article is not to resolve the debate about self-reported turnout measures. After all, we only have one type of turnout measure (self-reported), so we cannot compare the performance of different turnout measures in our models. However, we acknowledge that self-reports are imperfect measures of political participation. Although it could be useful to have validated measures of voter turnout (with additional measures, we could replicate our models across different dependent variables), Add Health has never validated its turnout measures. In addition, some studies have found that even validated turnout measures seem to suffer from severe problems.Footnote ¹⁴ Ultimately, we hope future studies will replicate this study using other datasets and different turnout measures.

An Initial Look at the Effect of Civic Education on Turnout

One obvious concern associated with examining the effect of civic education on voter turnout is that some students might choose to pursue more civic education than others. For example, parents who are civically engaged might encourage their children to seek out civics and social studies classes. Alternatively, some students might be predisposed to enjoy political and social issues because they like cognitive stimulation. As a consequence, these students may opt to enroll in more civics and social studies classes than their counterparts. While variables measuring civic education might appear to have an effect on voter turnout, they may actually be capturing unmeasured variables that are correlated with civic education and that also predict voter turnout (for example, parental socialization). As an initial way of addressing confounding, we develop models that include the measures of civic education described above as well as a wide range of control variables that capture individual and family attributes that should influence voter turnout and that might also be correlated with civic education. We control for respondent sex, age, race/ethnicity, frequency of religious attendance and cognitive ability. We also control for family background/experiences by integrating measures of parental education (mother's highest level of education), parental civic engagement (mother's participation in a civic group or club) and parental income.Footnote ¹⁵ Lastly, we include a measure of the respondent's highest level of education at the time when turnout is measured (either Wave III or Wave IV). This measure is designed to capture selection into or out of higher education after high school ends (Henderson and Chatfield Reference Henderson and Chatfield2011; Kam and Palmer Reference Kam and Palmer2008).

An initial look at the effects of civic education on voter turnout is presented in Table 1, which displays eight different models. For each dependent variable, we show the effect of civic education without control variables and then with controls, which investigates how the effects change after accounting for background characteristics.Footnote ¹⁶ Before proceeding, it is worth noting that because Table 1 entails multiple tests, it is valuable to correct the significance level to account for multiple testing. Here, we use the Benjamini-Hochberg procedure.Footnote ¹⁷ The basic idea behind this procedure is to order m p-values and then find the largest p-value that satisfies p_k≤ (k/m)a. This test and all tests with smaller p-values are declared statistically significant. Since we have two different dependent variables and there are twenty tests (civics measures) for each dependent variable, we implemented the Benjamini-Hochberg procedure on the twenty civics p-values in the Wave III models (Models 1, 2, 5 and 6 in Table 1) and then on the twenty civics p-values in the Wave IV models (Models 3, 4, 7 and 8 in Table 1). For the Wave III models, among the civics courses that are denoted as statistically significant in Table 1, only one of them becomes insignificant after correcting for multiple testing (civic skills development in Model 5). For the Wave IV models, all of the civics variables that are denoted as statistically significant remain significant after correcting for multiple testing.

Table 1.

Effect of civic education on voter turnout in adulthood

*p < 0.05, **p < 0.01, ***p < 0.001, two-tailed

In Models 1 and 3, the total number of social studies/civics courses a student takes has a positive and statistically significant effect (p < 0.001) on both measures. In Models 2 and 4, the effects remain statistically significant even after adding the set of individual and family controls, though the coefficients decrease slightly in both cases. Overall, we note that the substantive effects are fairly small. For instance, Model 2 indicates that an additional civics course in high school is associated with a 0.9-percentage-point increase in turnout in adulthood (it is worth noting that baseline turnout is fairly low). Thus, all else being equal, a person who took zero civics courses would have a predicted turnout rate of 39.39 per cent in adulthood and their turnout rate would be expected to increase to 40.24 per cent if they increased the number of civics classes to one. Models 5–8 report the results for the variables that count how many courses each respondent had from each category. Models 5 and 7, which show the effects of civics courses without controls, illustrate that a number of different course types have statistically significant effects on turnout.

In Models 6 and 8, which add control variables, several of the statistically significant effects dissipate, although a number of the categories still have significant effects. Model 6 has four statistically significant civics coefficients, and Model 8 has two. Put another way, of the eighteen civics courses in Models 6 and 8, 33 per cent have statistically significant effects. Courses on marginalized groups and political knowledge have an effect on the Wave III turnout measure, but these effects do not appear to persist over a longer period of time. Indeed, neither variable has a statistically significant effect on the Wave IV turnout measure, which was measured at least 6 years after high school.

The two types of courses that do seem to have enduring effects on voter turnout are experiential learning courses and those focused on multicultural/international topics. Both of these measures are statistically significant in the Wave III and Wave IV models that include controls (Models 6 and 8). Once again, though, the substantive effects are fairly small. For example, if we look at the multicultural/international measure in Model 6, the coefficient indicates that an additional course in high school is associated with an increase in turnout in adulthood of about 1 percentage point. Similarly, the coefficient for the experiential learning measure reveals that an additional course in this area in high school increases turnout in adulthood by about 3 percentage points.Footnote ¹⁸ As a final note, it is worth mentioning that across the models, there is little evidence that classes focusing on service learning, civic skills development and political issues increase turnout after we control for individual and family attributes. These seem like just the types of classes the resource model of participation (Brady, Verba and Schlozman Reference Brady, Verba and Schlozman1995; Verba, Schlozman and Brady Reference Verba, Schlozman and Brady1995) would expect to increase turnout.

Family Fixed-Effects Models

The models in Table 1 provide valuable leverage over the issue of confounding. However, it is possible to push our data a bit further to see whether the observed civic education effects discussed above hold up to additional scrutiny. Do the civics courses that have significant effects on turnout in adulthood reported in Table 1 persist in the context of family fixed-effects models? If they do, it may be a signal that these courses are particularly effective in elevating turnout in adulthood.

Because the Add Health study has a fairly large sample and contains a subsample of siblings within the same families, we can utilize family fixed-effects models.Footnote ¹⁹ We compare the estimated effect of civic education in a conventional cross-sectional model in which siblings are treated as individuals (without accounting for family) to the estimated effect in a fixed-effects model that accounts for all of the influences that are shared by siblings within the same family. Comparing the turnout of siblings with different levels of civic education but the same family background is as close as we can get to a controlled experiment. In this case, siblings from the same family serve as each other's control group. This approach allows us to control for all of the observed and unobserved family characteristics shared by the siblings, because only within-family variation is used to estimate the fixed-effects models.Footnote ²⁰ Yet this approach cannot estimate the effect of unobservable confounders that are not shared by siblings. The family fixed-effects approach has recently been used to study the relationship between voting and health (Burden et al. Reference Burden2016), education (Dinesen et al. Reference Dinesen2016; Gidengil et al. Reference Gidengil2019) and non-cognitive factors (Hillygus, Holbein and Snell Reference Hillygus, Holbein and Snell2016; Holbein et al. Reference Holbein2020). However, we are not aware of studies that employ this approach to examine the relationship between civic education and turnout.

While powerful, the family fixed-effects model rests on two important assumptions that, if violated, may lead to biased results. First, the model assumes that determinants of within-sibling differences in civic education are not correlated with voting. If there are unaccounted for sibling differences resulting from unshared experiences (for example, birth order, personality traits, cognitive ability) that positively influence both civic education and voting, the estimated effect of civic education on turnout will be inflated. Concerns about confounding, of course, are only relevant if we find a significant relationship between civic education and turnout based on the fixed-effects model.Footnote ²¹

The second assumption is the stable unit treatment value assumption (SUTVA). In our case, SUTVA requires that the civic coursework of one sibling only influences their voting behavior and not the voting behavior of another sibling. If one sibling's civics coursework also increases another sibling's likelihood of voting, this would bias the estimated relationship between civic education and turnout in the fixed-effects model downward. Our data contain twin and non-twin siblings; we would expect a greater chance of a SUTVA violation if we were focused only on twins since identical (monozygotic) twins typically report being close to one another and thus potentially influence one another (Dinesen et al. Reference Dinesen2016). We should also point out that concerns about SUTVA violations are less likely to be an issue for our Wave IV measure of turnout since it was elicited when respondents were between 25 and 34 years old – thus the spillover of civic education from one sibling to the next would have to persist for a very long time. It could also be the case that civic education for one sibling depresses the turnout of another sibling, but this seems highly unlikely. Below, we provide a preliminary test of this idea and find little evidence that this occurs.

Before analyzing the results, we note that for the fixed-effects analyses we specify linear models. We use this approach instead of logistic regression because in cases where families do not vary on the dependent variable (all siblings voted or none of them voted) they would be dropped when estimating the logistic fixed-effects models.Footnote ²² Using linear models lets us include these siblings in our analysis, which is an important advantage of this approach. We present the results of the fixed-effects models graphically in Figure 1.Footnote ²³ There are two panels – one showing the results for the Wave III turnout measure and one showing the results for the Wave IV measure. It is worth recalling that in Table 1 a total of eight civics measures were significantly related to turnout after controlling for other factors. Thus Figure 1 focuses on the effects of these measures. In each panel, we present ordinary least squares (OLS) estimates (controlling for age and sex) in which respondents are treated as individuals without regard to their membership in a family and familial factors are not taken into account, and the family fixed-effects estimates, which gauge the within-family impact of civic education on turnout. These are the estimates of primary interest, but it is useful to present the OLS estimates since they provide a sense of how the relationship between civic education and turnout changes after accounting for confounding factors rooted in the family. In short, the OLS estimates presented in Figure 1 are similar to the effects of civics courses shown in Table 1 (for the models that do not include control variables). It is important to point out one limitation of Figure 1: the samples used to estimate the effects shown in the figure are considerably smaller than the samples used in Table 1 since the figure only uses the subset of siblings in the sample. In Figure 1, the Wave III sample is comprised of 1,923 siblings and the Wave IV sample is comprised of 1,579 siblings. Thus, we acknowledge that the smaller sample sizes in the sibling fixed-effects analyses lead to reduced precision in our estimates compared to those in Table 1.Footnote ²⁴ However, we still believe it is worth examining the results of these models, even if they are taken as suggestive until similar studies can be conducted using larger sibling samples.

Figure 1.

Comparing effects of civic education in OLS and family fixed-effects models

There are a few things worth noting when looking at Figure 1. First, the civic education effects are fairly small in magnitude given the point estimates and the ranges of the confidence intervals. Secondly, most of the effects that remained significant in Table 1 after accounting for controls seem to dissipate once we account for family fixed effects (Wave III – total civics classes, experiential learning, marginalized groups; Wave IV – total civics classes, experiential learning). Although none of the civics variables is statistically significant at conventional levels in the fixed-effects models in Figure 1,Footnote ²⁵ the coefficient for courses on multicultural/international issues is close to statistical significance in both the Wave III and Wave IV fixed-effects models (p = 0.067 and p = 0.124 [two-tailed], respectively). Thus, of the nine different types of civics courses we analyze, these types of courses appear to be the most likely to be related to voter turnout in adulthood.

Overall, the results in Figure 1, when considered alongside those presented in Table 1, suggest that most civics courses do not have statistically significant effects on turnout in adulthood and that any effects that exist are unlikely to yield large increases in turnout in adulthood (as many have hoped).Footnote ²⁶ Above, we noted that sibling fixed-effects models come with limitations, such as decreased precision. Although our results are suggestive of null relationships between most civics courses and turnout in adulthood, we note that even if one considers the OLS estimates in Figure 1, which only control for age and sex (and are therefore likely to be liberal estimates of the effects), the civic education effects are very small, just as they were in our initial analyses. For example, the coefficient for multicultural/international courses in the Wave III model is 0.022, which means that an additional civics course in this area in high school is associated with an increase in turnout in adulthood of just 2.2 percentage points. Such an effect is fairly small given that some interventions that are far less expensive and time consuming than civics courses (such as postcards) can increase turnout by 2 to 8 percentage points (see, for example, Gerber, Green and Larimer Reference Gerber, Green and Larimer2008) and can have effects that persist for years (Davenport et al. Reference Davenport2010). It is also interesting to note that the civic education effects in Figure 1 are comparable to some of the socialization effects in Table 1. For instance, a one-unit increase in parental education corresponds to a 1.4-percentage-point increase in turnout in adulthood. Similarly, children whose parents are civically involved exhibit a 3.3-percentage-point increase in turnout when they reach adulthood compared to their counterparts. Thus civics courses, even when they work, do not appear to make up for differences in family background.

Overall, our takeaway is that while civics courses likely have null effects on turnout in adulthood, the best-case scenario is that civic education exerts very small effects overall. Ultimately, we hope future researchers will replicate our analyses using larger sibling samples. We are not aware of any other datasets that contain sibling samples, transcript-based civics measures and turnout measures, but we encourage the collection of such data. Although our results provide initial evidence that civics courses have little effect on voter turnout in adulthood, additional analyses will be helpful as political scientists, civic groups, policy makers and educators continue to think about the effectiveness of civic education in fostering voter turnout.