CIVIC EDUCATION AND POLITICAL KNOWLEDGE

One difficulty in garnering the resources to study civic education policy is that some possible outcomes (tolerance, efficacy, interest, political activity) risk ideologically tinged controversy among policymakers (Jamieson Reference Jamieson2013). Many are also difficult to measure. Fortunately, however, one outcome—political knowledge—is normatively desirable and empirically tractable.

Democracy requires an electorate with at least a modicum of political knowledge. Voters should know the fundamentals of how government works and be aware of the key people and debates featured in contemporary politics. While political scientists have long debated how knowledgeable the citizenry both is and ought to be (Achen Reference Achen1975; Converse Reference Converse and Apter1964; Delli Carpini and Keeter Reference Delli Carpini and Keeter1996; Hardin Reference Hardin2006; Lupia Reference Lupia1994; Page and Shapiro Reference Page and Shapiro1992; Popkin Reference Popkin1994), the disagreement is largely a matter of degree: how much voter knowledge does democratic competence require? Galston has summarized this literature by noting that “there are signs of an emerging consensus. Competent democratic citizens need not be policy experts, but there is a level of basic knowledge below which the ability to make a full range of reasoned civic judgments is impaired” (2001, 218–9).

Not only is there consensus on the need for basic political knowledge within the electorate, a robust literature has developed to measure it. Among youth, the federal Department of Education has periodically evaluated the state of political knowledge among 4th, 8th, and 12th graders with the National Assessment of Educational Progress (NAEP). While there will always be controversy over what comprises essential knowledge for informed citizenship, the designers of NAEP have found enough consensus to create high quality exams on the fundamentals of American government. Similarly, political knowledge is widely and effectively measured in mass surveys. Notably, Zaller argues that even short indices of factual political knowledge are a valid measure of political awareness, noting that “tests of knowledge are relatively immune to a social desirability response set” and “do not require respondents to estimate subjective behaviors or inner states” (Reference Zaller1992, 335).Footnote ⁴

Beyond political consensus and empirical tractability, political knowledge also stands on firm theoretical ground as arguably the outcome most likely affected by classroom civics instruction, for the same reason that we would expect that taking a chemistry course increases knowledge of chemistry. Civics courses may also affect other outcomes such as tolerance, efficacy, and a desire to participate in politics, but at a minimum increasing knowledge is presumably an objective shared by all teachers and policymakers. Delli Carpini and Keeter (Reference Delli Carpini and Keeter1996, 278) suggest that a base of political knowledge leads to the acquisition of still more information, producing a virtuous circle of ever more-informed voters. They also hypothesize that greater political knowledge is a first, and perhaps essential, step toward greater political participation (224–5).

CIVIC EDUCATION REQUIREMENTS

Many different policy levers might have an impact on the efficacy of civic education, but perhaps the most straightforward are states' requirements for civics. While all states include civics among their educational standards and most states have some sort of civics requirement for graduation from high school, specifics vary widely (Godsay et al. Reference Godsay, Henderson, Levine and Littenberg-Tobias2012). Such variation enables a comparison of different requirements. Furthermore, because states periodically change their civics standards and requirements—sometimes strengthening them, sometimes weakening—it is possible to test whether such changes have an effect on young people's political knowledge.

Owing to the rise in standardized testing across the United States, a significant body of research has developed to study whether external exams—and, specifically, “high stakes” exams that are a requirement for passing a grade or receiving a high school diploma—result in greater academic proficiency. While this area is rife with controversy and conflicting conclusions (Hursh Reference Hursh2005; Nichols, Glass, and Berliner Reference Nichols, Glass and Berliner2006; Supovitz Reference Supovitz2009), the balance of the extant research suggests that such exams lead to improved student performance (Carlson, Borman, and Robinson Reference Carlson, Borman and Robinson2011; Carnoy and Loeb Reference Carnoy and Loeb2002; Carnoy, Loeb, and Smith Reference Carnoy, Loeb, Smith, Carnoy, Elmore and Siskin2003; Dee and Jacob Reference Dee and Jacob2011; Diamond Reference Diamond2012; Gaddis and Lauen Reference Gaddis and Lauen2014; Hanushek and Raymond Reference Hanushek and Raymond2005; Rouse et al. Reference Rouse, Hannaway, Goldhaber and Figlio2013; Sims Reference Sims2013; Supovitz Reference Supovitz2009; Valenzuela Reference Valenzuela2005). There are potentially many, and mutually reinforcing, reasons for the effect of high-stakes exams, both salutary and worrisome. On the positive side, knowing they face a “day of reckoning,” students may work harder to learn a given subject. Similarly, an external exam may incentivize teachers to implement the most effective pedagogical practices and administrators to assign the most effective instructors. On the negative side, there is also a concern that these incentives are perverse and lead teachers to “teach to the test” at the expense of other skills and dispositions that foster civic engagement. Whether there is teaching to the test or not, any discussion of standardized exam results should include the caveat that such tests cannot fully reflect all that a student has learned.Footnote ⁵

In light of the past research into external exams in subjects other than civics, we hypothesize that state-mandated assessments in civics result in greater political knowledge both for students currently in high school and young adults in the years immediately following high school (age 18–24). Specifically, we expect that civics assessments have the biggest effect in states where they matter most—i.e., where they are a requirement for high school graduation—the incentive hypothesis. Footnote ⁶

If a civics assessment, or any aspect of civic education, leads to greater political knowledge, there is good reason to expect a greater impact among some groups than others. Past research into civics courses has found that, among the population as a whole, the civics curriculum has either no (Langton and Jennings Reference Langton and Jennings1968) or a modest (Niemi and Junn Reference Niemi and Junn1998) effect, although more recent studies of specific curricula have found larger effects (Facing History and Ourselves 2010; McDevitt and Kiousis Reference McDevitt and Kiousis2004; Kawashimi-Ginsberg Reference Kawashimi-Ginsberg2012). One argument for the limited effect of taking a civics class is that unlike many other subjects, knowledge about politics can be acquired through channels other than the classroom. Youth who are exposed to political discussion at home or through their friendship networks or who consume politically oriented news media are likely to soak up political knowledge (Hooghe and Dassonneville Reference Hooghe and Dassonneville2011; McIntosh, Hart, and Youniss Reference McIntosh, Hart and Youniss2007). Adolescents are, for example, much less likely to hear their parents discuss chemistry over dinner than politics.

By this logic it stands to reason that civics classes will have the greatest impact on young people with less exposure to political information at home and in their social networks—the compensation hypothesis. More precisely, we hypothesize a compensation effect where civic education is also subject to an incentive in the form of an external or “high stakes” civics exam. Notably, in the 1960s Langton and Jennings (Reference Langton and Jennings1968) found that while civics classes had no effect on the population as a whole, they did lead to increased political knowledge for African American students.Footnote ⁷ This was an era of widespread disenfranchisement for African Americans and so, Langton and Jennings hypothesized, black youth were less likely to have been politically socialized at home than were white students. Langton and Jennings were writing at the end of an era, as the Civil and Voting Rights Acts had just been enacted into law, ending formal disenfranchisement of African Americans. Today, voter turnout among blacks is nearly equal to or even exceeds that of whites (File Reference File2013). Latinos however, have a comparatively low rate of turnout and other forms of participation (García Reference García2011) which likely has consequences for the civic awareness of Latino youth. Many Latino youth are immigrants, children of immigrants, or surrounded by adult role models who are immigrants. Immigrants, whether naturalized citizens or not, are likely to be less familiar with the American political system than people who came of age in the U.S. (Callahan and Muller Reference Callahan and Muller2013). Owing to the concentration of immigrants within the Latino population, Latino youth in particular arguably have limited exposure to political information about U.S. politics and thus less fluency with the U.S. political process than non-Latinos. Thus, it is not just that Latinos themselves are often immigrants; it is also that Latino communities are less politically engaged overall. Accordingly, past research has shown political knowledge to be especially low among Latino youth (National Center for Education Statistics 2011; Torney-Purta, Barber, and Wilkenfeld Reference Torney-Purta, Barber and Wilkenfeld2007).Footnote ⁸

The compensation hypothesis thus leads to an expectation that today it is Latinos, and especially Latino immigrants, whose political knowledge is most likely to rise in states that have a civics assessment with academic consequences. In support of this expectation, Callahan and Muller (Reference Callahan and Muller2013) find a similar compensation effect when children of immigrants take social science classes because, as they put it, “children of immigrants are less likely to have parents who are familiar with or engaged in the U.S. political system” (2013, 9). They measured voter registration and turnout, but we hypothesize that the same applies to political knowledge.

In sum, we will test two hypotheses. First, the incentive hypothesis states that political knowledge will be higher among youth who live in states that have an external, consequential civics assessment. Second, the compensation hypothesis posits that consequential civics assessments have the biggest effect on the political knowledge of Latinos, and immigrants in particular.

We test these hypotheses with data collected from high school students, specifically the 12th grade portion of the 2006 and 2010 NAEP Civics test.Footnote ⁹ We first use NAEP data in a cross-sectional analysis to test the effect of civic education requirements in both 2006 and 2010. We then examine the effect of changes in those requirements over time by testing whether adding an assessment led to higher test scores and/or removing an assessment brought scores down. We then turn to the question of whether any observed effects persist beyond high school with analysis of a large national survey of 18–24-year-olds. Taken together, the differences between the studies serve to underscore the significance of the consistent findings.

ANALYSIS OF THE NAEP CIVICS TEST

Our analysis of the NAEP 2006 and 2010 test results rests crucially on coding states and the District of Columbia according to their requirements for civic education. To that end, we draw on data collected by researchers at the Center for Information and Research on Civic Learning and Engagement (CIRCLE), and classify states according to their civics requirements circa 2012 as well as changes in those requirements between 2004 and 2012 (Godsay et al. Reference Godsay, Henderson, Levine and Littenberg-Tobias2012). Table 1 displays states in four categories. First are states that strengthened their civics requirements between 2004 and 2012.Footnote ¹⁰ In some cases, this means they went from having no requirements to adding a civics course, or from requiring only a course to adding an assessment. Second are states that weakened their requirements, typically by dropping their assessment (although note that Iowa went from requiring a course to having no requirement). The third and fourth categories are states that did not change their requirements; the former's requirements are coded as strong (both a course and an assessment) while the latter's are weak (a course only). Note that the table provides two more pieces of information: (a) whether the only course requirement is in the broad subject of social studies, which typically includes government and politics but does not focus exclusively on it; and (2) whether the state assessment is a high school graduation requirement, or counts toward a grade in a civics course that is required for graduation (such as in Kentucky). Keep in mind that the requirements in the first two columns of the table are not necessarily limited to high school; assessments that are not a graduation requirement may be administered in middle school. It is unlikely, however, that assessments not required for graduation and administered four years prior to graduation will have an effect in the 12th grade; if courses and assessments apply to middle school, it should, if anything, sharpen our test of the incentive hypothesis.

TABLE 1.

State Civic Education Requirements in 2004 and 2012

^a The assessment is a graduation requirement or counts toward a grade in a civics course that is required for graduation.

^b Tennessee implemented an assessment in 2012, which was too late to affect any of the respondents in the Post-Election Survey.

Source: Godsay et al. (Reference Godsay, Henderson, Levine and Littenberg-Tobias2012).

We stress that our analysis tests the impact of state-level policies and not, as is more typical in studies of civic education, respondents’ self-reports about their own experiences in high school. Self-reports are subject to selection bias, as students with greater intrinsic interest in politics are more likely to take civics courses and to remember them as engaging. In contrast, policies set by the state are not subject to this sort of bias in selection or recollection. Nor is it likely that families choose to move into or out of a state because of its civic education requirements.

Another concern is the potential for the inconsistent implementation of a state's policies by districts or schools. While statewide exams are reliably administered to all students (barring outright fraud), the content of civics courses varies widely, as does the quality of instruction. Owing to these inconsistencies, consider the analysis of state-level policies to be analogous to the “intention to treat” (ITT) group in a randomized medical trial—i.e., subjects who are assigned to receive the treatment but who may or may not follow the prescribed protocol. Just as an ITT analysis estimates the aggregate impact of the treatment in question and is thus a conservative estimate of its effect, so is ours the lower bound of the effect civic education requirements have on political knowledge.

We first test our hypotheses with the 2006 and 2010 NAEP civics exam. As noted above, NAEP is an exam created and administered nationwide by the federal Department of Education to students in a representative sample of schools, both public and private. The NAEP program includes tests given to students in 4th, 8th, and 12th grade. For this analysis, we use the civics exam administered to 12th graders, under the assumption that this is the age group most likely to be affected by the prospect of a civics exam required for graduation from high school. This is also the grade at which students are most likely to take a civics course (Niemi and Smith Reference Niemi and Smith2001). It is important to stress that because NAEP is administered by the federal Department of Education and has no consequences for either students or their teachers, there is no concern about “teaching to the test.” In fact, the full content of the exam is not in the public domain and is thus inaccessible to teachers. Even if instructors wanted to teach to the test, they could not learn what was on the test in order to teach to it.

While it has many advantages, NAEP data nonetheless present some analytical challenges.Footnote ¹¹ For one, NAEP uses a complex “block” design whereby no single student completes the entire 150 or so test items, which makes it impossible to calculate a straightforward “percentage correct” for each student. For our analysis, we use the student's standardized test score, as calculated by NAEP staff using item response theory, as the dependent variable. Unfortunately, this results in a nonintuitive metric. Therefore, to provide a clearer interpretation of the results, we recode the variable to have a standard deviation of 1 and a mean of 0. Because schools in the study were selected using a multistage probability sampling design, our models estimate sampling variance using a Taylor series expansion method.Footnote ¹² The standard errors account for the design of the sample, as the estimation specifies the primary sampling units and the clusters within the PSU, as well as an overall weight to ensure that the sample reflects the national population of high school seniors.

We start with a cross-sectional analysis, whereby we use regression analysis to examine the relationship between states’ civic education requirements and students’ NAEP scores in both 2006 and 2010. Drawing on the data displayed in Table 1, we code states as being in one of four categories: (1) No Civics Requirement Footnote ¹³ ; (2) a civics course is required, but there is no assessment; (3) a civics course and assessment are both required, but the assessment is not a high school graduation requirement (Low-stakes Civics Assessment); (4) a civics course and assessment are both required, and the assessment is a requirement for graduation, or it counts toward a grade in a course required to graduate (High-Stakes Civics Assessment). Each is coded as a binary variable. The omitted variable is (2)—a course but no assessment—because this is the modal category. Thus, the coefficients for these variables are interpreted as the effect compared to a state that requires a course but has no assessment.

We then move to a more stringent test by examining the effect of changes in civic education requirements. Does the addition of requirements lead to greater political knowledge? Does removing them lead to less knowledge?

The models include an array of control variables in order to account for potentially confounding factors at the individual, school, and state levels.Footnote ¹⁴ Since socioeconomic status correlates so strongly with both political knowledge and academic performance, the models have multiple individual-level controls for socioeconomic level. The students reported these on the background questionnaire that accompanies the NAEP exam. One is parental education, coded as the highest level of education obtained by either parent. Another is the number of books in a student's home, a proxy for intellectual stimulation in the home. Others include whether the student is eligible for a free or reduced price lunch, and whether there is a computer in the home. In addition, the models control for gender, race and ethnicity (specifically whether a respondent is Latino, unless the model is for Latinos only), the frequency that a language other than English is spoken in the home, and whether the student is classified as an English language learner. At the school level, the models control for whether the school is public, private and Catholic, or private but not Catholic, as well as the racial (percent African American) and ethnic (percent Latino) composition of the school population, as reported by school administrators. At the state level, the model controls for aggregate socioeconomic status with the mean per-student expenditure on education and median household income. Because of our focus on Latinos (see below) the model controls for the percentage of Latinos in the state. To control for average academic performance, it also includes the state's mean score on 2009 12th grade NAEP reading and math exams.Footnote ¹⁵ The dependent variable closely approximates a continuous variable and so we employ OLS as an estimator while accounting for the complex design of the sample, as described above.

The models displayed in Table 2 test the incentive hypothesis by examining whether students in states with more stringent civic education requirements have higher scores on the NAEP exam. In the models for all students, the control variables generally behave as expected. Measures of socioeconomic status in the home lead to higher scores, while minority groups (except for Asians/Pacific Islanders) score a little lower on the exam. In column 1 of the table we see that, in 2006, civics assessments also lead to better performance on the NAEP exam. Compared to students in states with a course but no assessment, those in states with a course and a low-stakes exam score 0.08 of a standard deviation higher (p = 0.10).Footnote ¹⁶ The effect for a High-stakes Civics Assessment is comparable—0.11 of a standard deviation, and a p value of 0.05. Column 3 shows the same pattern in 2010. There is a statistically significant effect for states with civics courses that both do and do not have a high-stakes assessment (for both, p < 0.05), with similar effect sizes for both. Figure 1 presents the results of the models in Table 2 in graphical form.

TABLE 2.

The Effects of Civics Requirements on Political Knowledge in 2006 and 2010: OLS Regression, Accounting for Complex Sampling Design

Note: The omitted civics variable is (2)—a course but no assessment. *p < 0.10; **p < 0.05; ***p < 0.01.

Source: National Assessment of Educational Progress Civics Assessment, 2006 and 2010; for civics variables see Table 1; demographic data is from the 2010 U.S. Census.

FIGURE 1.

Effects of Civics Requirements

Notes: Omitted category is a civics course without an assessment. Bars represent the 95% confidence interval.

In other words, analysis of cross-sectional data in 2006 and 2010 provides support for the incentive hypothesis. For the general population, having a civic education requirement of some type leads to more political knowledge.

The models in columns 2 and 4 test the compensation hypothesis by restricting the analysis to Latino students. In doing so, we follow the advice of Masuoka and Junn, who convincingly argue that testing theory about differential effects by race requires estimating separate models for each racial group (2013, 8). The hypothesis predicts that any incentive effects will be greater for Latinos, owing to their relative lack of familiarity with the American political system.Footnote ¹⁷ Note that, when the model is restricted to Latinos only, the control variables follow in much the same pattern as for the general population, although fewer of them reach statistical significance (no doubt because of the smaller sample size). The results do not confirm the compensation hypothesis in 2006, but provide some suggestive support in 2010. In 2010, we see higher scores (p < 0.05) for Latino students in states that have a civics course and an assessment, with a comparably-sized effect for both (0.27 for a low-stakes assessment, 0.22 for high stakes). Even though these effects are larger in magnitude than the comparable effects for non-Latinos in 2010, further analysis (not shown) finds that the difference in the effects for Latinos and non-Latinos does not reach statistical significance and so the compensation hypothesis is not fully confirmed in 2010.

Note that there is also a positive and significant effect for the category of No Civics Requirement (0.27). Before concluding that this is evidence that civic knowledge improves in the absence of a civics requirement, recall that in 2010 this variable only describes Iowa, which has a strong tradition of local control over education. Further research should be directed toward the study of civic education at the local level in Iowa, which apparently is successful among Iowa's growing Latino population.

While a first step, the cross-sectional data provide only a weak test of our hypotheses. Even with a plethora of controls for state-level characteristics, the states may differ in other, unmeasured ways that affect their students’ performance on the NAEP exam. A stronger test of the two hypotheses examines changes within the states, which essentially holds the unmeasured features of each state constant.Footnote ¹⁸

The incentive hypothesis predicts that adding civics requirements will lead to an increase in political knowledge, and thus that their corresponding variables will have positive coefficients. Specifically, we would expect adding a high-stakes assessment to have the biggest effect. Likewise, states that removed civics requirements might see a drop in their students’ scores. On the other hand, only low-stakes exams were eliminated, which suggests little or even no effect on civic knowledge. As explained previously, the compensation hypothesis suggests that any observed effects will be larger for Latinos.

States are coded according to whether they Added a High-Stakes Civics Requirement, Added a Low-Stakes Civics Requirement or Removed a Low-Stakes Civics Requirement. The baseline (and modal) category is states that made no change.

Column 1 of Table 3 presents the results, while Figure 2 summarizes the key findings. These models have the identical set of control variables as the cross-sectional ones, save for the addition of a dummy variable for the year (2010=1). Again, the control variables contain no surprises. When we turn our attention to the variables recording change in the states’ civics requirements, we see no effect for the general population; the incentive hypothesis does not find support. However, when the model is restricted to Latino students (column 2), we see strong evidence for the compensation hypothesis.Footnote ¹⁹ Adding a high-stakes assessment corresponds to an increase of 0.40 of a standard deviation, a sizable effect that is highly significant (p < 0.000). Furthermore, an interactive model (not shown) confirms that this effect is significantly different than the effect for adding a high-stakes requirement for non-Latinos.

TABLE 3.

The Effects on Political Knowledge of Adding or Removing Civics Requirements, 2006–2010: OLS Regression, Accounting for Complex Sampling Design

Note: The omitted civics variable is states that made no change. *p < 0.10; **p < 0.05; ***p < 0.01.

Source: See Table 2.

FIGURE 2.

Effects of Adding and Removing Civics Requirements, 2006 to 2010

Notes: Omitted category is no change in civics requirements. Bars represent the 95% confidence interval.

Adding a low-stakes civics requirement has a positive effect, but the confidence interval is so wide that it is nowhere near statistical significance. Removing an assessment leads to a drop in the Civics Scale, but it too is far from significant. Recall, though, that the only civics assessments to be removed were inconsequential for students’ graduation from high school (i.e., they were low stakes). Note that the compensation effect is relatively large in substantive terms. By comparison, the effect of increasing one unit on the scale measuring the number of books in the home has a comparable, albeit slightly smaller, effect (0.24 of a standard deviation).

To summarize our findings thus far, the cross-sectional analyses in 2006 and 2010 provide support for the incentive hypothesis in both years, and for the compensation hypothesis in 2010. In a stronger test of the hypotheses, the analysis of change in civics requirements is consistent with the compensation hypothesis, as the civic knowledge of Latino students increased substantially in states that adopted a high-stakes civics assessment.

AFTER HIGH SCHOOL

For all the strengths of the NAEP analysis, it leaves unanswered the question of whether the effects of civic education requirements endure beyond high school or if they are merely transitory and thus soon fade away. To see whether effects persist after high school, we turn to a survey of 18–24-year-olds conducted in 2012, known as the Commission on Youth Voting and Civic Knowledge Post-Election Survey 2012 (for brevity, we will refer to it as the Post-Election Survey). Although not a panel survey, it does enable a test of whether civic education requirements in the states where respondents went to high school have a bearing on their political knowledge in 2012.Footnote ²⁰

The primary independent variables are the same as in the earlier cross-sectional analysis; the coding reflects the civic education requirements in the state in which respondents attended high school in the year that they were 18 years of age.Footnote ²¹ The dependent variable, however, is different. While the NAEP exam contains scores of items designed to test different areas of political knowledge, the Post-Election Survey was limited to a brief six-item index, containing items common to many such surveys (Delli Carpini and Keeter Reference Delli Carpini and Keeter1996). They cover both topical knowledge (e.g., which party has more members of the House of Representatives) and the fundamental “rules of the game” in the American system of government (e.g., share of congressional votes needed to override a presidential veto).Footnote ²² The average score was 3 out of 6 correct. Roughly 9 percent got none of them correct, while 4 percent had a perfect score.

Undoubtedly, a more in-depth and nuanced set of items (like those found on the NAEP test) would be more informative. However, using an index with such limited variance makes for a conservative test of our hypotheses, as it biases the analysis against finding a signal amid the noise.Footnote ²³

The model includes controls for an array of individual-level characteristics, including race/ethnicity (African American, of Asian descent, or Latino), gender, and age. We control for educational attainment in two ways. First, because a growing literature indicates that relative educational attainment matters more than absolute levels of education (Campbell Reference Campbell2009; Nie, Junn, and Stehlik-Barry Reference Nie, Junn and Stehlik-Barry1996; Persson Reference Persson2011) we calculate the respondents’ education level relative to the mean of their age cohort.Footnote ²⁴ Second, since these respondents are 18–24 years old, we also include a dummy variable for whether they are currently enrolled in school. As with the NAEP models, we control for intellectual stimulation in their formative years with an item that measures the number of books in the respondent's childhood home.Footnote ²⁵ In addition, we include a control for immigration status, which was unavailable in the NAEP data. Specifically, the model accounts for whether respondents themselves or at least one of their parents are immigrants to the U.S.Footnote ²⁶ The model also controls for an array of state-level variables. These include the general academic performance in the state where the respondent attended high school, as measured by the state's mean NAEP scores in math and reading, and that state's average per-pupil expenditure on education. The model also includes median household income of the state where the respondent currently resides. Because the survey was conducted during a presidential election year, we also control for the political environment of respondents’ current state, in case greater salience for the presidential race or a stronger participatory culture fosters greater political knowledge. We thus include a state's degree of electoral competition in 2012, and turnout among 18–24-year-olds in 2010. We again employ OLS as an estimator, estimate robust standard errors, and cluster the standard errors by state.Footnote ²⁷

As in the earlier models, the incentive hypothesis predicts that respondents who attended high school in states with civics requirements and, in particular, a high-stakes civics assessment, will have greater political knowledge. The results are displayed in column 1 of Table 4. The control variables conform to expectations; in particular, a higher level of education and greater intellectual stimulation in one's childhood home lead to greater knowledge, while women and minorities have lower levels of knowledge. State-level variables generally have no effect. Most importantly, the incentive hypothesis finds little support. While the coefficient for a consequential assessment is positive, its p value is only 0.22.

TABLE 4.

The Effects on Political Knowledge of Civics Requirements after High School (18–24-year-olds): OLS Regression

Note: The omitted civics variable is (2)—a course but no assessment. *p < 0.10; **p < 0.05; ***p < 0.01.

Source: 2012 Post-Election Survey of U.S. citizens aged 18–24 (Levine Reference Levine2012); for civics and demographic variables, see Table 2.

We turn next to the compensation hypothesis by limiting the analysis to Latinos, as displayed in column 2 of Table 4. The hypothesis finds strong support, as also shown in Figure 3. Latino respondents who attended high school in a state with a consequential civics assessment score 0.52 higher on the political knowledge index (p < 0.01). That translates to a nine percentage point increase. By comparison, this is a slightly bigger effect than being enrolled in school (0.41). Expressed another way, a consequential assessment has an effect size of 0.32 of a standard deviation.

FIGURE 3.

Effects of High-Stakes Civics Assessments in High School (18-24 Year-Olds).

Note: Bars represent the 95% confidence interval.

Unlike the NAEP data, the Post-Election Survey enables a more precise test of the compensation hypothesis by limiting the analysis to respondents who are first or second generation immigrants (column 3 of Table 4).Footnote ²⁸ Obviously, this category includes people of all ethnicities and excludes some Latinos. That is, not all immigrants are Latinos, and not all Latinos are immigrants. We again see a positive, significant effect for a consequential assessment that is slightly larger than the effect for all Latinos—0.64 (p < 0.05) on the knowledge index, 11 percentage points, and 0.40 of a standard deviation.Footnote ²⁹ Finally, we subject the compensation hypothesis to one further test by limiting the analysis to Latino immigrants. The results provide resounding support—a high-stakes assessment produces a 1.11-point increase on the political knowledge index (p < 0.001), which translates to an increase of 18.5 percentage points, and 0.68 of a standard deviation.Footnote ³⁰ An interactive model (not shown) that explicitly compares the effect of a high-stakes assessment for Latino Immigrants versus the general population confirms that the difference in the effects is statistically significant (p < 0.01).Footnote ³¹

As still further support for the compensation hypothesis, when the model is restricted to Latinos who are not immigrants, the coefficient for a High-stakes Civics Assessment is nearly the same as for the general population (0.14) and far from statistical significance (p = 0.55) (results not shown). In other words, simply being a Latino does not lead to an effect for a consequential assessment; rather, the effect is only for immigrants.