Hostname: page-component-6898cfcb55-cbk24 Total loading time: 0 Render date: 2026-11-12T00:03:14.704Z Has data issue: false hasContentIssue false
  • English
  • Français

How Deliberation Affects Policy Opinions

Published online by Cambridge University Press:  01 November 2004

JASON BARABAS
JASON BARABAS
Affiliation:
Harvard University and Southern Illinois University
Rights & Permissions [Opens in a new window]

Abstract

Theorists argue that deliberation promotes enlightenment and consensus, but scholars do not know how deliberation affects policy opinions. Using the deliberative democracy and public opinion literatures as a guide, I develop a theory of opinion updating where citizens who deliberate revise their prior beliefs, particularly when they encounter consensual messages. A key aspect of this model is that opinion strength moderates the deliberative opinion change process. In two separate propensity score analyses using panel survey data from a deliberative forum and cross-sectional surveys, I show how deliberation and discussion both affect opinions toward Social Security reform. However, deliberation differs from ordinary discussion in that participants soften strongly held views, encounter different perspectives, and learn readily. Thus, deliberation increases knowledge and alters opinions, but it does so selectively based on the quality and diversity of the messages as well as the willingness of participants to keep an open mind.

Information

Type
ARTICLES
Information
American Political Science Review , Volume 98 , Issue 4 , November 2004, pp. 687 - 701
Copyright
© 2004 by the American Political Science Association

What happens when people deliberate? We are told that deliberation improves democracy by improving citizens. This is the spirit of the argument that Page (1996) makes when he claims that “public deliberation is essential to democracy, in order to ensure that the public's policy preferences—upon which democratic decisions are based—are informed, enlightened, and authentic” (1). Yet, informed, enlightened, and authentic opinions are in short supply. Ample evidence suggests that Americans know little about politics (Delli Carpini and Keeter 1996), are misinformed (Kuklinski et al. 2000), or possess unstable attitudes (Converse 1964) that deviate from otherwise fully informed preferences and votes (e.g., Althaus 1998 and Bartels 1996). Against such bleak assessments, scholars offer deliberation as a way of rejuvenating citizens and, ultimately, democracy.

Unwavering faith in deliberation is puzzling because scholars have not clarified how deliberation works. On this point Page and Shapiro (1999) state, “We consider it to be very much an open question just how well deliberation works, by what mechanisms, under what circumstances.” They emphasize that we need to learn “… how [real-world] deliberation affects public opinion, and how real-world deliberative processes can be made better” (111–12; see also Mendelberg and Olseke 2000, 169). Thus, scholars think deliberation is important (Elster 1998) but know little about it. In the absence of a theory and empirical testing, we cannot be sure that deliberation enhances opinion rationality, elevates citizen competence, or enriches democracy.

In what follows, I identify two goals of deliberative democracy, enlightenment and consensus, which motivate my inquiry into how deliberation affects public opinion. Based on these goals and the public opinion literature, I construct a theoretical account that directs attention to the strength of opinions that citizens possess prior to deliberation and the diversity of the discussants. I go on to test hypotheses generated from this theory in two empirical studies to show how deliberation affects knowledge and attitudes as well as the extent to which deliberation differs from nondeliberative experiences.

THE GOALS OF CONSENSUS AND ENLIGHTENMENT

Democratic deliberation is often depicted as cooperative search for agreement. All participants need not actually reach the same decision every time, but they are supposed to aim for mutual understanding while casting their personal interests aside. Cohen (1989) argues that deliberative democracy “aims to arrive at a rationally motivated consensus” (23, emphasis in original), although it is important to note that Cohen (1996) believes that nonconsensual mechanisms, such as majority rule, are necessary parts of the deliberative system. Consensus is also an underlying theme for one of the best-known deliberative theorists, Jürgen Habermas (1989). As Hardin (1999) notes, “Often, Habermas implicitly supposes that there is an underlying consensus that deliberation will discover” (106). Although later Habermas (1997) moves away from strict definitions of consensus when considering how difficult it is to find and maintain in pluralistic societies, he, along with Gutmann and Thompson (1996), argues that deliberation leads to agreement on matters where disagreement once existed.

While it is a goal of deliberation, consensus may not be desirable if social inequalities mean that groups who lack power or who deliberate in nontraditional ways feel intimidated or coerced (Fraser 1991; Sanders 1997). Submissive consensus is clearly undesirable, but even absent any overt coercion, concurrence-seeking behavior may also a product of what Janis ([1972] 1982) calls “groupthink,” which he argues can occur even in groups composed of experts. Thus consensus itself need not be a good thing. Indeed, consensus might be a pejorative concept if one considers the work of social psychologists who study the reasons people provide when describing their own decision-making (Levine, Halberstadt, and Goldstone 1996; Wilson and Schooler 1991). According to these studies, individuals cannot reliably account for their opinions, so deliberation and group decision-making may lead to perverse outcomes (also see Asch 1951).

This means that to be desirable, deliberation should improve knowledge so that participants come not only to a consensus, but also to an enlightened view concerning the problem at hand. In other words, deliberation ought to provide consensus and enlightenment. Citizens should base their opinions on accurate factual information about the political world. The notion that truth emerges from discussion is quite common, and starting from Plato and Aristotle, the Western political tradition has produced elaborate versions of it (Manin 1987). Classic liberals, such as Mill ([1859] 1947), refined the idea of the truth emerging in a free marketplace of ideas as long as competition and diversity are present. Young (1996) expresses this idea in more general terms when she states that a deliberative encounter where citizens express, question, and challenge differently situated knowledge “… adds to the social knowledge of all the participants” and that “this greater social objectivity increases their wisdom for arriving at just solutions to collective problems” (128).

Thus, proponents of deliberative democracy value both consensus and enlightenment. In fact, some theorists, like Robert Dahl (1989), count enlightened understanding as an essential ingredient of democratic processes, further connecting deliberation and democracy. The relevant question now becomes how enlightenment and consensus operate in the deliberative context. These themes are related to the theory of deliberative opinion updating presented in the next section. As I will argue, individuals who deliberate gain information that they use to construct postdeliberative opinions. However, deliberation carried out as theorists envision it differs from most other opinion change processes in at least two important ways. In true democratic deliberation, individuals do not cling to their prior opinions. Citizens open themselves up to the possibility of change while being exposed to information and arguments that may be dramatically different from what they ordinarily encounter. Open-mindedness and diversity are central concepts that emerge from the literatures on deliberation and public opinion.

DELIBERATION AND PUBLIC OPINION

Some of the best-known research on deliberation is based on “deliberative polls” that James Fishkin and his colleagues conducted in the United States and in England by gathering randomly selected citizens in small groups to discuss political issues. A primary concern in these studies is whether opinions change in the aggregate. If response frequencies differ after deliberation, then the deliberative poll is typically hailed as a success (Fishkin 1995, 167).

The balance of the early aggregate evidence indicates that participants in the U.S. deliberative polls learned and formed significantly different opinions after the forums on at least some of the issues considered (Fishkin and Luskin 1999). However, opinion change does not always take place. An earlier Granada 500 deliberative poll in Britain found few instances of opinion change (Denver, Hands, and Jones 1995). Similarly, a meta-analysis of seven National Issues Forums in the early 1990s found that deliberation improved participants' political sophistication even though changes in aggregate opinions were negligible (Gastil and Dillard 1999; see Crosby 1995 for similar research on Citizens Juries, which also predate the deliberative polls). Thus, the empirical evidence on deliberation presents a mosaic of opinion changes in some cases, the absence of changes in others, and occasional variations in the underlying components of attitudes without visible effects on aggregate opinion results.

Although these studies occasionally document opinion effects, at times the evidence remains inconclusive, and most empirical analyses of deliberation do not explain the patterns of opinion change or the lack of change. That is, researchers actively seek, occasionally find, and enthusiastically celebrate evidence of changes in aggregate survey frequencies without attending to the underlying opinion processes at work. These same inconsistencies are found in research on group decision-making from other disciplines. For example, social psychological studies of group and interpersonal influence often yield seemingly contradictory evidence of attitude polarization in some instances or “risky-shifts” where participants become consensual but more extreme in their judgments (Vinokur and Burnstein 1974, 1978). The question becomes how to reconcile the empirical evidence with deliberative theory. If deliberation truly holds such promise for democracy, then we must attempt to understand how it influences political opinions and the ways in which it differs from other opinion change processes.

Opinion Strength and Diversity

The starting point for this study comes from Zaller and Feldman's portrayal of survey responses as inherently variable. Their core insight is that instead of representing fixed positions, survey responses at any given moment are central tendencies within a distribution of many possible responses (Zaller and Feldman 1992; Feldman 1995). Opinion distributions are made up of “considerations,” which are defined as reasons that might induce an individual to decide a political issue one way or another. According to Zaller (1992), opinion change takes place when individuals choose to accept or reject the arguments they encounter, which results in a store of considerations that are available later when evaluating political issues. Variations in elite discourse, individual differences in attention to these elite messages, and individual differences in political predispositions jointly determine the distribution of public opinion.

Although still made up of considerations, opinions formed during deliberation are unique in two ways. First, an important procedural condition under which democratic deliberation takes place encourages participants to keep an open mind.1

In his definition of democratic deliberation, John Gastil (2000) claims that participants must “consider carefully what others say” (22). Careful consideration of the viewpoints of others implies that citizens keep an open mind and will not reject arguments outright. Urging citizens to remain impartial is precisely what happens when jurors are instructed to keep an “open mind” (see Ward 2004 for jury admonitions in California). See Correll, Spence, and Zanna 2004 for experimental evidence on the relationship between keeping an open mind and sensitivity to persuasive communications. Open-mindedness or rigidity may manifest in certain personality types (e.g., Adorno et al. 1950). It is worth noting that encouraging citizens to weaken their opinions can lead to undesirable consequences. It is the highly opinionated who often participate in politics, so reducing opinion strength might reduce participation.

Open-mindedness can be defined in various ways, but the key aspect is a genuine willingness to consider the opinions of others by relaxing the intensity with which one holds a given view. Opinion change, as will be argued shortly, is virtually impossible if individuals retain strongly held views.2

A strongly held opinion is not the same thing as strongly agreeing or strongly disagreeing with a policy option. As will be argued shortly, a strong opinion in the Bayesian sense is one that is highly certain (or “precise”), no matter where it might lie on a continuum.

Such a statement may not seem novel, especially considering the literature on attitude strength (Petty and Krosnick 1995), but an important though undervalued feature of deliberation is that citizens are obligated to open up to the possibility of attitudinal change.

Assuming that individuals maintain an open mind, then whether they actually change their views after deliberating about politics depends on a second important procedural condition: diversity. Disagreement is a defining principle for deliberative theorists (Gutmann and Thompson 1996). Citizens do not ordinarily converse with dissimilar discussants, either by choice or because the supply of discussants is limited (Huckfeldt and Sprague 1995). In other words, most individuals do not seek out discussants who hold divergent political views (e.g., Mutz 2002 and Mutz and Martin 2001). Even when citizens are around people with whom they disagree, norms against discussing contentious issues like politics often prevail (Crandall, Eshleman, and O'Brien 2002). Deliberation is structured so that people with diverse views commingle.

These two conditions—open-mindedness and diversity—are procedural requirements of deliberation that differentiate it from ordinary discussion and other opinion change processes. However, relaxing strong opinions and assembling a diverse group result in opinion change only to the degree that deliberators strive for consensus. It is at this point where the two goals of deliberative democracy become important again. Unless deliberators reach a consensus, especially an enlightened one, they will just talk past each other, resulting in a cacophony of divergent views. Although diversity and consensus are seemingly at odds, it is this tension, along with relaxing strong views, that makes deliberation special and sets it apart from what citizens ordinarily encounter.

To understand how deliberation works, we must determine whether the diversity and open mind conditions are met. If they are, then we can focus on the degree of consensus in the deliberative messages. That means for predicting opinion change we need to know where deliberators start (i.e., their predeliberative opinions) and how strongly they hold their views, but the main concern is whether deliberation leads to verbal agreement. These are the three most important factors, and as one might imagine, they can combine in many different ways. Thus, we need a way to formalize the process to generate expectations. As argued next, the subtleties of deliberative opinion change emerge when viewed as an instance of Bayesian opinion updating.

Opinion Updating

If we treat individual opinions and deliberative messages as distributions, then Bayes' rule can be used to understand what happens when distributions merge. That is, the theorem can be used to describe what happens when an opinion is revised based on new information, where “information” is used generically to refer to anything from facts to arguments. The basic intuition is that individuals have prior opinions (“priors”) and that these are updated with new information, yielding revised opinion preferences. New information and the quality of that information, as defined by its clarity (also referred to as its “certainty” or “precision”), determine the placement and precision of subsequent (“posterior”) opinion responses. Predictions about opinion change or, more accurately, predictions about changes in the distribution of considerations given particular configurations of prior opinions and new information can be derived from Leamer's (1978) theorems on combining two distributions using Bayes' rule. It is not necessary to master the mathematical formulas to understand the basic logic, and others already have illustrated the how the process works (Alvarez 1997, 25–51; Jones 2001, 94–96). The central idea is that predeliberative opinions blend with new information obtained via deliberation (or for that matter from any other message-generating process) to determine the postdeliberative opinions jointly.

Opinion updating depends on the precision of the information (i.e., in Bayesian parlance, whether the new data about the world are “informative” or “clear”). When a person encounters precise information that differs from a prior perception, it should result in opinion change (e.g., a person who previously “agreed” with a proposition might “disagree” on hearing precise disagreement messages). It is possible, however, to update an opinion without changing it, such as when new information reinforces preexisting beliefs (e.g., a person who “strongly agrees” encounters “strongly agree” messages). In these situations, posterior opinions become more certain. Thus, opinion updating does not always lead to different opinions in the sense of moving to a different point on a number line. It depends on where individuals start as well as the direction and precision of the deliberation.

The idea of opinion or belief updating is not new (Bayes [1763] 1958) and already plays a role in political science (e.g., Achen 1992, Alvarez 1997, and Bartels 1993). Opinion change understood as a Bayesian process systematically accounts for the way in which individuals combine newly encountered information with their past knowledge of the political world. It is a natural extension of Zaller's (1992) Receive–Accept–Sample model and the notion of distributions of considerations. Not only opinions are treated as distributions of considerations, but so are the messages that a person encounters. The advantage of using the Bayesian paradigm in this context is that it accounts for the many possible combinations of predeliberative opinion profiles and messages. When participants are urged to keep an open mind and when they encounter different perspectives, deliberation can lead to opinion change in situations where participants come to a verbal consensus. Consensual deliberation is important in that it serves as a way of identifying message precision and thus structures predictions about deliberative opinion change.

Hypotheses

Elsewhere (Barabas 2000) I simulate the effects of clear or unclear messages on distinctly liberal, moderate, and conservative starting positions to deduce two important points underscoring the logic of Bayesian updating in a deliberative setting. The first deduction is that significant opinion change is probable only for individuals who do not hold strong opinions. However, weak opinions are a necessary but insufficient condition. Whether opinions change depends on another factor. Opinions shift in the aggregate only when diverse deliberators with weak views come to an agreement. Therefore, we should see changes in policy opinions when deliberators have relaxed their prior opinions and when there is verbal consensus. In other words, when open-minded and diverse deliberative participants agree, we should expect opinion movement in the direction of the deliberative agreement. When deliberators are divided in their comments (i.e., no verbal consensus), then opinions will not change significantly in the aggregate (i.e., no opinion consensus). Temporally, I assume that this process runs from relaxing prior opinions to deliberation to posterior opinion formation.3

To be deliberative opinions, we focus on opinions after deliberation. In other words, citizens reason through potentially conflicting views before arriving at their opinion (Chong 1993). In transcripts from the forum described later, citizens talked through multiple sides before constructing their opinions.

As stated earlier, consensus is desirable to the degree that it is accompanied by enlightenment. This means that any investigation of consensual movements in opinions will first have to consider the nature of the deliberation and whether citizens learn. Thus, the hypotheses guiding this inquiry are as follows: (H1) citizens who deliberate will significantly increase their proportions of correct responses on objective knowledge questions, (H2) aggregate opinion will move significantly in the direction of consensual deliberation, and (H3) opinion strength will moderate opinion change (i.e., individuals who possess weak [strong] prior opinions will change the most [least]).

However, recall that in ideal deliberative situations participants relax strongly held views. That means if we see evidence of verbal consensus, consensual opinions should follow for all deliberative participants. Thus, H2 and H3 are interrelated and are contingent on the degree of discursive consensus, with consensus operationalized here as at least two-thirds of the directional statements on one side of a policy option.4

Gastil (2000) defines consensus in deliberation as a two-thirds majority (151). Although it did not characterize the deliberation of Study 1 in the next section, it is possible, of course, that deliberators could remain locked in disagreement before ultimately arriving at a consensual judgment. In this case, a better measure of the consensus would be the direction of agreement in the end rather than the balance of pro and con statements. More research is needed on what constitutes a consensual (or precise) message.

In sum, I will look for gains in knowledge and movements in opinions that are a function of the deliberation itself as well as the characteristics of the deliberative participants.

These hypotheses have clear ties to the enlightenment and consensus goals of the deliberative democracy literature. I use them to guide an empirical study in the following section, where we have detailed information on the actual content of the deliberation as well as measures of knowledge and opinions before and after the deliberation. However, to illustrate how deliberation differs from other opinion change processes, I also investigate what happens when citizens discuss the same issue in situations that do not typically utilize the procedures that deliberative theorists recommend. More to the point, the second study demonstrates what happens when individuals discuss Social Security without the procedural encouragement to relax the strength of their prior opinions and when they may not have access to high-quality information or a diverse set of discussants. It is impossible to make directional predictions (H2) for the second study because there is no way to observe the discussion or the discussants. It is, however, still possible to observe the end product—the postdiscussion opinions—so we will search for evidence of learning or opinion change via ordinary discussion.

STUDY 1: DELIBERATION IN A SOCIAL SECURITY FORUM

A deliberative forum called Americans Discuss Social Security (hereafter ADSS) was held on May 30, 1998, in Mesa, Arizona, a suburb of Phoenix. Sponsored by the Pew Charitable Trusts as a way for citizens to discuss possible Social Security policy reforms, the forum embodied the open-minded, diverse, constraint-free exchange found in many scholarly treatments of face-to-face deliberation. It was held on a Saturday and 408 citizens gathered in a large convention center with approximately 50 round tables, each with chairs for 8 to 10 participants and a moderator. The entire forum lasted approximately five hours; participants were not compensated monetarily. After initial introductory comments from the forum organizer, Dr. Carolyn Lukensmeyer, participants were given a chance to discuss values relating to Social Security, hear experts present facts about program operation and benefits, and deliberate about various policy options intended to reform Social Security.

The goal of keeping an open mind was invoked early and often. In refering to the procedural goals of the forum, Lukensmeyer said, “One is that every person's voice is heard who's in this room. The second is that everyone is listened to and respected.” Just before the deliberation commenced, Lukensmeyer paraphrased William Grieder (1992) to the effect that “Creating a positive future begins in human conversation.” Particularly relevant to the present study, Lukensmeyer told the forum participants, “That's the spirit of open-heartedness and open-mindedness that we hope can encourage your tables today, to really begin the conversation about the future of Social Security with the belief that what I say and what we say today at this table has the possibility of making a difference” (emphasis added). Thus, ADSS participants were explicitly encouraged to abandon strongly held views and to remain open to the possibility of attitude change.5

I assume that forum participants relaxed their prior opinions (i.e., opened their minds) as instructed.

ADSS had provided participants with informative materials before they attended the forum. ADSS also attempted to educate participants during the forum itself. Issue immersion included almost all the components of the Fishkin model—balanced briefing materials selected by a group of experts, intensive small group discussions, and the opportunity to question experts—but unlike the deliberative polls, participants did not hear politicians debating opposing positions. Instead, they heard nonpartisan background presentations on Social Security delivered by representatives of the Social Security Administration and by representatives of the American Academy of Actuaries.

Another difference was in the way the forums were held and recruited. Whereas Fishkin (1995) invited a nationwide random sample of respondents to travel to Austin, Texas, for the National Issues Convention, ADSS tried to recruit a representative cross section of the local population. A public affairs firm sent out invitations to a large random sample of registered voters in the county where the forum was held. Most forum participants were recruited via this random selection mechanism. Forum organizers tried to boost attendance through media and Internet ads inviting members of the public to attend. In an effort include local groups, organizers asked a wide variety of local organizations to invite participants.6

Fewer than 10% of those who were invited or attended were recruited by interest groups.

The fact that forum organizers used partially nonrandom selection methods, coupled with the likelihood of selective participation among those who were recruited randomly, creates several potential methodological problems that will be addressed after a preliminary discussion of the survey data.

The panel data for Study 1 are drawn from pre- and postforum surveys administered by the National Opinion Research Center (NORC) to three groups: (1) those who participated in the forum (“attended”), (2) those who were invited to the forum but did not attend (“invited”), and (3) a random sample of people in Maricopa County, Arizona (“random sample”), where the forum was held.7

The overall response rate was 52% for the respondents who were invited to the forum (241 of 462) and 45% for the random sample (148 of 331).

Table 1 shows the various treatment and control conditions.

If assignment to three subgroups had been random, it would have been possible to compare the survey responses of those who attended to the responses of those who did not attend to learn the effects of forum participation. However, in quasi-experimental settings with nonrandom assignment to treatment, simply analyzing differences between treatment and nontreated “control” groups may lead to biased estimates of treatment effects if individuals self-select into the treatment and comparison groups (Shadish, Cook, and Campbell 2001).

Propensity scores, defined as the conditional probability of assignment to treatment given a set of covariates, can be used to reduce bias by balancing the nonexperimental treatment and control groups (Rosenbaum and Rubin 1983). The logic underlying propensity score analysis is that while comparisons between the forum participants and the two control groups may not be valid if they differ in many important respects, it is possible to match those who have a high likelihood of attending and who actually attended with those who had a similar likelihood of attending but who did not attend. Matching techniques reduce bias by adjusting estimates of the treatment effect as if the whole study were a randomized experiment (Dehejia and Wahba 1999).

The variant of propensity score analysis employed here uses a probit model on the entire preforum sample to predict the likelihood of forum attendance for any given respondent. The linear predictions from this probit model are then used as propensity scores to help match participants with otherwise similar nonparticipants; once matched, the values of the dependent variables for the treated and control cases are compared to discern the effects of the treatment (i.e., matched cases are alike in every observable way except one case received the treatment and the other did not). Propensity score analysis yields readily interpretable results—mean values for the treated group and the matched control group—with clear substantive meaning for the theory of opinion updating presented earlier. One-to-one nearest-neighbor matching with replacement was used here, but the results are highly robust when compared against seven other propensity score matching techniques in the literature. A methodological appendix is available upon request for readers interested in the sensitivity analyses, bias reduction properties, and technical details underlying this method.8

One-to-one nearest neighbor matching with replacement is the default option in the Stata propensity score matching routine developed by Edwin Leuven and Barbara Sianesi. The probit models yielding the propensity scores in both studies contain standard demographic variables (e.g., income, education, age, race, sex) as well as relevant measures of knowledge, attitudes, trust, participation, and salience.

See Barabas (2004) or Imai (n.d.) for applications of propensity score methods in political science.

We can observe the effects of deliberation only for the respondents who participated in both survey waves. That is, it is possible to study the effects of deliberation only by concentrating on the 389 respondents in the t2 (postforum) survey because these are the only observations after the forum deliberation. Fortunately, the respondents who participated in the t1 and t2 surveys do not differ significantly from those who participated only at t1 (preforum) according to Sherman's (2000) test for ignorable attrition. However, Sherman's test revealed violations of the missing at random assumption for the respondents who participated in either wave, suggesting that item nonresponse was nonrandom for certain questions. In the statistical analyses to follow, missing demographic responses in the form of “don't know” were imputed with King et al.'s (2001) Amelia program to create multiple estimates for missing demographic responses.

The Content of the Forum Deliberation

The effects of deliberation should be related to its content. As such, the best way to begin is to examine the deliberation itself. Although deliberation was unconstrained in the sense that participants were allowed to dwell on any subject, the substantive policy conversations centered on three Social Security options according to a content analysis of every audible statement made at a sample of six randomly selected tables where participants were seated (Cook and Jacobs 1998).9

Comments were coded as statement clusters, which are defined as “a specific idea, topic, comment, or question made by anyone at the forum” (Cook and Jacobs 1998, 5). Overall, there were 4,213 statement clusters at the forum, distributed over a universe of 15 values, 23 fact, and 28 possible Social Security reforms identified by Social Security experts. Intercoder reliability analyses on a random sample yields kappa values of .80 (“excellent”) for policy options and .64 for directionality (“good”).

The three most prominent options were the proposals (1) to raise the limit on taxable wages above the 1998 level of $68,400, (2) to reduce the cost of living adjustment (hereafter “COLA”), and (3) to allow individuals to invest part of their Social Security tax contributions in their own privately controlled savings accounts (see the supplemental methodological appendix or Gramlich 1998 for background information on the reform options).

Because message direction is so important when predicting deliberative opinion change, policy comments were separated into “support,” “neutral,” and “oppose” categories. Almost half of the statements were directional. Of these directional comments, there was a high degree of verbal support for the proposal to raise the earnings cap (38% of comments for it versus 7% of the comments against it). The COLA was discussed positively almost twice as often as negatively (30% to 16%). Opinions on the privatization option were more balanced (22% to 25%).10

Citizens made 121 statements devoted to the earnings limit option, 116 on the proposal to allow workers to invest, and 43 on the cost of living adjustment option. It was not possible to link statements made at the individual tables to particular survey respondents, but the content analysis can be used to characterize the deliberative environment and structure predictions.

If the ratio of positive to negative comments is used as an indicator of message direction and consensus, the picture that emerges of is one of clear support for increasing the earnings cap, a vague preference for lowering COLAs, and mixed but mildly unsupportive comments regarding partial privatization. The content analysis guides expectations for the opinion analyses, where the key research question will be whether opinions shift in these same directions. Using a definition of consensus based on verbal agreement of two-thirds of the directional comments, the only clear expectation is for higher levels of support on the earnings limits option. Aggregate-level opinions should not be expected to have shifted significantly on the other two policy options.11

Some scholars worry that experts at deliberative forums bully citizens into supporting certain views (Page and Shapiro 1999). That was not the case for the ADSS participants. According to the content analysis, elites who spoke at the forum (e.g., moderators, actuaries, Social Security personnel, etc.) were almost exclusively neutral with one exception. On the accounts option elites spoke against it in approximately 20% of their statements, while making no positive statements. Thus, if there was an elite message, it was to oppose the accounts proposal. However, as we shall see later in the opinion analyses, most citizens resisted this message.

It is important to recognize that facts were also a part of the deliberation at the ADSS forum. Of the almost two dozen facts that coders identified, most concerned basic information about benefit structures, the Social Security budget, life expectancy, and solvency issues. Many of the facts mentioned at the forum appeared on the survey. The expectation tested here, as with the policy comments, is that increases in knowledge (or “enlightenment”) are a product of the entire deliberative experience.

Knowledge and Opinions Before and After the Deliberative Forum

The effects of deliberation should be evident through analysis of differences in the responses of the forum and matched control group samples. However, even in the absence of an identifiable treatment, simply answering the same survey question more than once can alter responses (see, for example, “regression artifacts” or “testing effects” in Shadish, Cook, and Campbell 2001, 57–60). In an attempt to be conservative and account for these potential problems, I calculated the (quasi-)experimental effect as the difference between the pre- and the postforum survey measures for the group that attended the deliberative forum subtracted from the difference in the pre- and postforum scores of the matched comparison group members.12

The net quasi-experimental effect is (Otreatment group, postdeliberation -Otreatment group, predeliberation) − (Ocontrol group, postdeliberation − Ocontrol group, predeliberation), where matched control cases are drawn from the invited-only and random sample groups and where O signifies survey observation.

Figure 1 illustrates the mean proportions of correct, incorrect, and don't know responses on the battery of knowledge items administered to the respondents (see Mondak 2000 on the importance of separating knowledge into these categories).

The Effects of Deliberative Forum Participation on Correct, Incorrect, and Don't Know Respones to Social Security Knowledge Questions

The changes in knowledge reveal evidence of learning. In Figure 1A, forum participants are represented by the black line with circles at the ends, whereas control group respondents are depicted by the gray line with diamonds at the endpoints. The forum group began with a lower proportion of knowledge than the matched control group, at .57 versus .61, although the groups were statistically equivalent as a result of the matching procedure (t112=0.99, p<.33).13

Differences between the forum and control groups at the t1 preforum survey are statistically insignificant (p>.05) based on $t$-tests for all the knowledge and opinion measures.

After the forum, deliberative participants increased their proportion of correct answers on the Social Security knowledge items to .77, compared with a more modest increase to .70 for the control group (t112=−2.06, p<.05). While the absolute gain for the forum group was .20, taking into account the smaller increase for the control group, attributable to the information packet and increased awareness of the knowledge battery items, the overall effect was a more modest +.10 (after rounding), which is still a statistically significant increase in knowledge from deliberation (Hotelling's F2,110=19.9, p<.01). Thus, the evidence supports H1 relating to a gain in information for the deliberative forum participants.

Figure 1B and C depict changes in the “incorrect” and “don't know” proportions for the deliberative participants and their matched control groups. Forum participants nearly halved their proportion of incorrect responses, from .32 to .17 (t112=−7.40, p<.01). The matched control group also reduced its proportion of incorrect responses, but the reduction was less dramatic, from .30 to .20 (t112=−4.35, p<.01). After taking the gains for the comparison groups into account, there were .05 net reductions in incorrect and don't know responses for the attendee group.14

Overall, the net reduction in the proportion of incorrect responses was .05 (the difference of −.15 and −.10), which was a statistically significant reduction (Hotelling's F2,110=57.9, p<.01). Figure 1C depicts a similar .05 net reduction in the proportion of don't know responses that was also statistically significant (Hotelling's F2,110=5.0, p<.01) based on a reduction for the forum group (.10 to .06) and a small increase in don't knows for the matched control group.

Overall, then, before the forum the two groups were indistinguishable. Afterward, the citizens who attended the deliberative forum were much more likely to know the correct answers to six specific pieces of factual information on the Social Security program and less likely to offer incorrect or don't know responses.15

The learning effects were not driven by the expert commentary or the informational packet. The experts made only 5% of the factual statements related to items tested on the surveys; in contrast, citizens made four times as many factual statements with direct relevance to the survey. Overall, citizens accounted for roughly three-quarters of all coded factual statements at the forum. Similarly, analyses in the supplemental appendix show that the forum learning was above and beyond what was provided in the informational packet (i.e., the same learning patterns appear when comparing the invitees to attendees, who both received the packet, so the only difference was the forum). Finally, the propensity score matching model includes a measure of how closely individuals read the packet so the groups have already been balanced on this dimension (i.e., I compare those who read it and attended with those who read it but did not attend, so any effects are above and beyond those attributable to the information packet).

According to the opinion change hypothesis (H2), attitudes should shift in the direction of deliberative consensus. As the content analysis indicated, the only area of clear verbal agreement beyond the two-thirds threshold was on the earnings limit option, with its more than five-to-one ratio of positive to negative comments. Therefore, we should see significant changes in favor of the policy option to increase the amount of earnings subject to Social Security payroll taxation past the (1998) level of $68,400. Beyond that option, the comments were not consensual by the definition adopted here, so H2 predicts no change toward the reform options of reducing the COLA and allowing workers to invest.

The three policy options displayed in Figure 2 show precisely this pattern. Forum participants increased their support for raising the amount of earnings subject to Social Security payroll taxes from .60 to .72, compared with a small decrease for the control group from the preforum to the postforum measures. The experimental effect depicted in Figure 2A is +.14, which is highly significant (F2,67=3.93, p<.05). Thus the opinions of forum participants coalesced on the issue where they found common ground during the deliberation.

The Effects of Deliberative Forum Participation on Three Social Security Reform Preferences

The other two policy options are shown in Figure 2B and C. Forum participants became more supportive of the option to reduce cost of living adjustments, moving from .29 to .31, but the control group showed a larger gain, starting at .35 and ending at .41. Consequently, the overall experimental effect is actually −.03 since the forum group did not move as much as the control group. That this difference is insignificant (F2,67=1.43, p<.25) is neither surprising nor unexpected. A similar picture emerges for the option to allow workers to invest. Forum participants went from a .61 average level of support to .53, while the control group declined from .56 to .49. The net effect after rounding was −.02, a difference that misses the p<.05 standard, although not by much (F2,67=2.59, p<.09). I expected negligible opinion changes on the partial privatization option given the tone of the policy deliberation. In fact, the actual shift was small. These insignificant changes are also taken as support for H2. Deliberative consensus gives rise to consensual opinions; deliberative disagreement has no significant effect on attitudes in the aggregate.

Thus far I have considered only deliberative message characteristics. In instances where there is a verbal consensus, all deliberative participants, having entered into the deliberation with an open mind, should move in the same direction. When a deliberative message is not clear, though, participants cannot follow the deliberative consensus because it does not exist. In these situations, those with weak prior opinions might respond to any hint of a verbal judgment (remember that the deliberation on the COLA and privatization options, while not consensual, still had some discernible direction) but those with strong opinions should resist or even move in the opposite direction. The bar graphs in Figure 3 provide support for the third hypothesis (H3) regarding opinion change for those with weak opinions.

The Effects of Deliberative Forum Participation by Strength of Prior Opinion

Figure 3 shows the net experimental effects for each policy options in the black bars. On either side of these bars are the effects for forum participants who held strong views and those who had relatively weak prior opinions, where strong opinions are operationalized as having a high degree of preforum knowledge.16

Measuring opinion strength (or certainty) is notoriously difficult (see Alvarez 1997, 53–75). Lacking more direct measures, I operationalized opinion strength in Study 1 by the level of preforum knowledge where those who are above the median on knowledge before the forum are classified as having strong (or “informative” in Bayesian terminology) opinions. In their discussion of the defining features of attitude strength, Petty and Krosnick (1995) write, “… The more knowledgeable a person becomes about an attitude object, the harder it will likely be to change his or her attitude toward the object…” (4). Similarly, knowledge is used to characterize opinion strength in Bayesian models (Alvarez 1997, 44).

The striking result for the proposal to raise the earnings subject to Social Security taxation is that participants with strong and weak opinions moved almost equally; the difference between the low and the high groups is not statistically significant (p<.33). Such movements across all groups of participants provide support for H3 regarding the persuasive effects of consensual deliberation for participants who engage in it with an open mind.

The two other policy options display a different and theoretically relevant pattern: Those with weak opinions moved with the prevailing direction of the deliberation, while strong-minded citizens tended to move the other way. This shows up on the COLA option, where participants with weak opinions (i.e., low in predeliberative knowledge) became more supportive, +.11, while those with strong opinions moved in the opposite direction. These movements offset each other in the aggregate (as indicated earlier, the .03 drop was insignificant), although they clearly differed from one another (p<.01). A similar pattern recurs for the partial privatization option. Only those with weak opinions responded to the slightly negative signal on this policy option. Those with strong attitudes moved in the opposite direction, though the differences between the subgroups are not significant (p<.78).

The small sample sizes and deliberative ambiguity suggest caution in interpreting these last results. However, the results show that researchers interested only in aggregate change might miss important subgroup movements that further illuminate how deliberation works. When deliberation is consensual, all participants are moved by it roughly equally. When participants fail to reach a consensus, only those without hardened views are persuaded, while strong-willed participants move not at all or in the opposite direction. Given the range of views articulated on both the COLA and the private accounts options, participants may have come to their own individual judgments, but because there was no consensus among all the ADSS participants, the effects washed out in the aggregate (see Druckman and Nelson 2003 for more on the power of deliberative dissent). I will put these findings into perspective in the Conclusion, after considering discussion of Social Security in nonforum settings.

STUDY 2: DISCUSSION OF SOCIAL SECURITY

Although field experiments provide analytical leverage on causality, they often leave questions about whether findings generalize to other contexts. That seems particularly true of the ADSS forum. Most citizens never participate in a forum devoted to public policy reforms. However, they do discuss politics. This happens in everyday life when friends, family, or coworkers chat about politics informally (e.g., Cramer-Walsh 2003 and Mansbridge 1999). To what extent does informal, “ordinary” discussion affect knowledge and opinions? Investigating these questions requires similar measures of Social Security discussion, knowledge, and attitudes.

Fortunately, such data exist. Working in conjunction with the ADSS organizers, Princeton Survey Research Associates (PSRA) conducted four random digit dial surveys to learn more about policy preferences and discussion of Social Security. The first survey began before the ADSS forum in March of 1998 and the last took place during May of 1999.17

The Roper Center for Public Opinion Research has sampling information and question wording for these surveys. A methodological appendix is available upon request for coding details.

The only way we know that survey respondents engaged in discussion is through their responses survey questions on their political discussion habits. Specifically, respondents were asked, “Thinking about the last month or so, which of these national issues, if any, have you discussed with your friends, neighbors, family members, or co-workers?” In the analyses that follow, the discussion of Social Security question is used as the principal independent variable of interest in assessing how nonforum Social Security discussion affects knowledge and support for the same three policy options considered in Study 1.

The directional aspects of the hypotheses cannot be tested because it is impossible to know what was said when respondents report discussing Social Security. Instead, the point is to see whether citizens who discussed Social Security learned and whether their opinions differed from those who did not discuss Social Security. Because the PSRA surveys include a question explicitly asking about opinion strength, I can segment the sample to provide suggestive evidence regarding the third hypothesis. The value of this study is that while citizens are discussing the Social Security issue, it is plausible to assume that most are not doing so while having been encouraged to keep an open mind and most probably are not interacting with discussants who will convey a viewpoint much different than their own. In short, two of the most important procedural conditions of deliberation are not typically present. Thus, observing a similar act—discussion—allows us to isolate the unique aspects of deliberation.18

To facilitate comparisons between Study 1 and Study 2, I use propensity score matching techniques for the PSRA data too, although bivariate probit models were also considered (Greene 2000, 849–56) to account for potential endogeneity (i.e., discussion influencing opinions or opinions stimulating discussion; see, for instance, McLeod, Scheufele, and Moy 1999). See footnotes 20 and 21 for more.

The Effects of Discussion on Knowledge

Discussion of Social Security is fairly common: When the four PSRA surveys asked respondents whether they had discussed Social Security or other major issues during the prior month, almost 70% said they had done so in March of 1998; four months later, the level of Social Security discussion dropped to 63% where it stayed through February of 1999 before falling to 56% in May of 1999.19

Regarding the validity of the self-reported discussion questions, it is reassuring to learn that discussion varies across issues and over time for seven issues in the PSRA surveys. Reflecting their prominence, education and health care are two other issues that are discussed almost as regularly as Social Security. The next most discussed items are tax and drug abuse issues, with just over half of the respondents reporting that they discussed these two topics during the last month. The environment is a discussion topic for about 47% of the respondents. Campaign finance discussion is less frequent (only 11–14%). In contrast, 80% claimed they discussed a highly salient issue like presidential impeachment. Viewed collectively, the patterns show variation in discussion habits and heighten confidence in these measures.

Did discussion of Social Security influence political knowledge? It appears not. When respondents who claimed to have discussed Social Security are compared to otherwise similar citizens who did not discuss Social Security, no significant differences emerge in the overall levels of knowledge for the two groups using t-tests on the means.20

Maximum likelihood models that account for the endogeneity of discussion show a significant increase in the proportions of correct and incorrect answers on an eight-item battery of Social Security facts (see the supplemental appendix for details). So, from a different methodological vantage point, citizens might learn as they discuss Social Security, but they also take in some inaccurate information too. Either way, the results are not the unambiguously positive story associated with forum deliberation.

This is an important finding because the forum deliberation had purely positive learning effects; discussing Social Security had no significant effect on levels of domain-specific Social Security knowledge.

Opinion Updating through Discussion of Social Security

Did nonforum discussion influence policy preferences? Yes; after matching survey respondents by their propensities to discuss Social Security, nonforum discussion of Social Security powerfully influenced two of the three policy options. Figure 4 shows the significant effects of Social Security discussion on support for raising the limits on the amount of income taxed for Social Security purposes as well as preferences toward partial privatization through individual accounts. Three opinion profiles appear in each panel, one for all respondents and two others to compare the effects for those with weak or strong views on Social Security. In Figure 4A, the overall effect of discussion is positive and significant. There is significantly (p<.01) more support for earnings limits (a mean of .64) among those who discuss Social Security than those who do not (.60).

The Effects of Discussion on Social Security Reform Preferences by Strength of Opinion

A key variable of interest is opinion strength. How did opinion strength moderate the effects of Social Security discussion? (The opinion strength question was, “How strong are your opinions about changing Social Security—very strong, somewhat strong, not too strong, or not strong at all?”).21

Respondents with strong opinions are those who rate the strength of their Social Security attitudes above the median (very/fairly strong). The propensity score matching model using a probit specification with one-to-one nearest-neighbor matching appears in the supplementary methodological appendix. In the maximum likelihood analyses, which also appear in the appendix, an interaction term identifies respondents with strong views who discuss Social Security. The ordered probit coefficients (standard errors) for the discussion strength interaction are −.10 (.01) for raising the earnings limit and .00 (.01) for the COLA. Bivariate probit models for the partial privatization (the only policy option where endogeneity appears to be an issue) yielded insignificant discussion and discussion×strength interaction terms.

Figure 4A shows how the effects of discussion are exaggerated for the weak opinion group (.66, compared to .61 in the control group), while the difference between the control and the discussion groups for strong opinion respondents is insignificant (.62 versus .59). The same pattern characterizes Figure 4B, except that discussion now leads to less support for partial privatization. Those who discussed Social Security were less willing to allow individuals to invest their retirement contributions in private accounts (a mean of .60 for the control group versus .57 for the discussion group; p<.05), an effect that holds for respondents who possessed weak views but not for those who claimed to have strong views.

This striking pattern of results lends additional support to the idea of opinion updating as well as the superiority of deliberation over ordinary discussion. Without the encouragement to soften their views, merely talking about Social Security does not help respondents with strong views update their opinions. The encouragement to keep an open mind that citizens received at the deliberative forum enabled them to relax their prior opinions so that opinion change was possible if citizens encountered precise (i.e., consensual) messages, which was the case with the earnings limit option. Moreover, in Study 1, ADSS respondents learned a lot about Social Security, far more than those who did not attend the forum and significantly more than citizens who discussed Social Security in nonforum settings.

The underlying conclusion is that deliberation leads to new, enlightened judgments. Discussion also affects policy attitudes, but only for those who have not already made up their minds and without the benefit of learning. Opinion strength matters since citizens engaging in ordinary discussion lack a procedural reminder to relax strongly held opinions. When all who discuss Social Security move in the same direction, it could signal discursive consensus, but it is not an authentic deliberative consensus in the sense of starting off in disagreement and moving closer together. Discussants might achieve verbal agreement easily because it was already there to begin with or because people are socialized to avoid controversial subjects. However, these speculations are based on prior studies (e.g., Mutz 2002 and Mutz and Martin 2001; cf. Huckfeldt, Johnson, and Sprague 2004) rather than the results reported here, for we could not observe the nonforum dialogue.

At the very least, then, ordinary discussion does not perfectly mimic deliberation. Discussion and deliberation differ in that deliberation, as democratic theorists envision it, takes place using procedural requirements that diverse participants relax their strongly held views. Deliberation is an enlightened and open-minded search for consensus amid diverse participants. Ordinary discussion can lead to similar conclusions, as it did on the earnings limit options. In fact, an optimistic reading of the support for the earnings limit option in the cross-sectional data might suggest that even ordinary discussion can serve as a heuristic, enabling people who lack deliberative opportunities to come to deliberative conclusions. However, discussion can also lead to opinion change on some issues—as it did on partial privatization—that were not observed to the same extent in the deliberative forum. The inconsistency of the findings in the two studies on the partial privatization option suggests that deliberation and discussion really do differ in important ways.

CONCLUSION

Many of us have been told or urge others to keep an open mind. Keeping an open mind, along with exposing ourselves to new information and diverse perspectives, is the essence of deliberation. It is what separates deliberation from discussion and other opinion influences. In two different studies, I have shown that citizens learn when they deliberate but not when they discuss politics. Deliberation is unique in that citizens discard their inaccurate factual perceptions as well as rigidly held political views. Deliberation represents an opportunity for opinion change, in the spirit of enlightenment and consensus, but there are no guarantees. The direction and magnitude of change depend on the deliberation and the procedural conditions.

The effects of the deliberation observed here—the substantial increases in Social Security knowledge and support for raising the limit on taxable earnings—might be a result of this particular deliberative forum. Another manifestation of deliberation in another place at another time might yield different results. Thus, I make no claim that deliberation leads to strictly liberal or strictly conservative preferences. Furthermore, there is no way of knowing whether these effects will endure. Yesterday's posteriors become today's priors. Whether deliberative judgments last depends on whether citizens encounter new and precise information that leads them to revise their opinions once again.

Scholars who study deliberation should continue to probe the underlying opinion processes at work as well as the procedures under which it takes place. For example, if deliberation is delegated to the media (Page 1996) or entrusted to political institutions like the Senate (Bessette 1994), are the diversity and open-minded conditions met? If not, then it will be important to discover how closely mediated forms of deliberation approximate democratic deliberation as theorists envision it, especially the degree to which the public benefits from alternatives to face-to-face deliberation. Formal deliberative forums, or at the very least settings that embody the principles of deliberative democracy, are needed to help encourage individuals with strong views to open themselves up to the possibility of change. Assuming that deliberation occurs under these conditions, citizens can form thoughtful opinions, but once again this depends on the clarity and quality of the deliberative messages.

Proponents of deliberative democracy, like Gastil (2000), want to see deliberation used more often. Gastil has devised innovative ways to include the results of deliberative forums on election ballots and referenda to help citizens make sound political decisions. In an attempt to reinvigorate the public sphere or recreate the fabled New England townhall meetings, philanthropic foundations sponsor civic engagement forums and media organizations experiment with public journalism. As these findings demonstrate, such attempts may prove worthwhile because deliberative opinions are different and more informed. In many ways, deliberative democracy in practice appears to live up to its theoretical promise.

Until recently the normative and empirical literatures on deliberation have evolved separately. As researchers continue to bridge this divide, the account of opinion updating presented here may have to undergo further revision. Nevertheless, theorists will benefit from this empirical study of deliberation, while researchers interested in political behavior can profit from a deep understanding of the core principles underlying democracy. What all can take away from the lessons learned here is that diversity and keeping an open mind, which are hallmarks of democratic dialogue, can help citizens transform potentially unreflective preferences into enlightened deliberative judgments.

A previous version of this paper was presented at the Center for the Study of Democratic Politics (CSDP) at Princeton University. I appreciate the help I received from the CSDP director, Larry Bartels, and the CSDP seminar participants. In addition, I received insightful comments and assistance from Jonathan Best, Dennis Chong, Tom Cook, Kathy Cramer-Walsh, Pat Dorsey, Charles Franklin, Tobin Grant, Bill Greene, Susan Herbst, Michael Herron, Jennifer Jerit, Kim Kochanek, Jim Kuklinski, Carolyn Lukensmeyer, Sally Murphy, Dennis Plane, Ben Page, and Lori Weber. This project was made possible by the generosity of Fay Lomax Cook and Larry Jacobs. I sincerely appreciate all their advice and support. The Pew Charitable Trusts funded the deliberative forum and surveys used in this research.

References

Achen Christopher H. 1992. “Breaking the Iron Triangle: Social Psychology, Demographic Variables, and Linear Regression.” Political Behavior 14 (September): 195– 211.Google Scholar
Adorno Theodor.W., Else Frenkel-Brunswik, Daniel J. Levinson, and R. Nevitt Sanford. 1950. The Authoritarian Personality. New York: Harper.
Althaus Scott L. 1998. “Information Effects in Collective Preferences.” American Political Science Review 92 (September): 545– 58.Google Scholar
Alvarez R. Michael. 1997. Information and Elections. Ann Arbor: University of Michigan Press.
Asch Solomon. 1951. “Effects of Group Pressure on Modification and Distortion of Judgment.” In Groups, Leadership and Men, ed. Harold Guetzkow. Pittsburgh, PA: Carnegie Press.
Barabas Jason. 2000. “Americans Discuss Social Security: How Deliberation Affects Public Opinion.” Ph.D. diss. Northwestern University.
Barabas Jason. 2004. “Virtual Deliberation: Knowledge from Online Interaction versus Ordinary Discussion.” In Democracy Online: The Prospects for Democratic Renewal Through the Internet, ed. Peter Shane. New York: Routledge, 239– 52.
Bartels Larry M. 1993. “Messages Received: The Political Impact of Media Exposure.” American Political Science Review 87 (June): 267– 85.Google Scholar
Bartels Larry M. 1996. “Uniformed Votes: Information Effects in Presidential Elections.” American Journal of Political Science 40 (February): 194– 230.Google Scholar
Bayes Thomas. [1763] 1958. “An Essay Toward Solving a Problem in the Doctrine of Chances.” Philosophical Translations of the Royal Society, 330– 418. (Reprinted, with biographical note by G. A. Barnard, in Biometrika 45: 293–315.Google Scholar
Bessette Joseph M. 1994. The Mild Voice of Reason: Deliberative Democracy & American National Government. Chicago: University of Chicago.
Chong Dennis. 1993. “How People Think, Reason, and Feel about Rights and Liberties.” American Journal of Political Science 3 (August): 867– 99.Google Scholar
Cohen Joshua. 1989. “Deliberation and Democratic Legitimacy.” In The Good Policy, ed. Alan Hamlin and Philip Pettit. Oxford: Basil Blackwell, 17– 34.
Cohen Joshua. 1996. “Procedure and Substance in Deliberative Democracy.” In Democracy and Difference, ed. S. Benhabib. Princeton, NJ: Princeton University Press, 95– 109.
Converse Philip E. 1964. “The Nature of Belief Systems in Mass Publics.” In Ideology and Discontent, ed. David Apter. New York: Free Press.
Cook Fay Lomax, and Lawrence R. Jacobs. 1998. Deliberative Democracy in Action: Evaluation of Americans Discuss Social Security. Report to the few Charitable Trusts. Evanston, IL: North-Western University Institute for Policy Research.
Correll Joshua, Steven J. Spencer, and Mark P. Zanna. 2004. “An Affirmed Self and an Open Mind: Self-affirmation and Sensitivity to Argument Strength.” Journal of Experimental Social Psychology 40 (May): 350– 57.Google Scholar
Cramer-Walsh Katherine. 2003. Talking about Politics: Informal Groups and Social Identity in American Life. Chicago: University of Chicago Press.
Crandall Christian, Amy Eshleman, and Laurie O'Brien. 2002. “Social Norms and the Expression and Suppression of Prejudice: The Struggle for Internalization.” Journal of Personality and Social Psychology 82 (March): 359– 79.Google Scholar
Crosby Ned. 1995. “Citizens Juries: One Solution for Difficult Environmental Questions.” In Fairness and Competence in Citizen Participation: Evaluating Models for Environmental Discourse, ed. O. Renn, T. Webler, and P. Wiedemann. Boston: Kluwer, 157– 74.
Dahl Robert A. 1989. Democracy and Its Critics. New Haven, CT: Yale University Press.
Dehejia Rajeev H., and Sadek Wahba. 1999. “Causal Effects in Nonexperimental Studies: Reevaluating the Evaluation of Training Programs.” Journal of the American Statistical Association 94 (December): 1053– 62.Google Scholar
Delli-Carpini Michael X., and Scott Keeter. 1996. What Americans Know About Politics and Why It Matters. New Haven, CT: Yale University Press.
Denver David, Gordon Hands, and Bill Jones. 1995. ”Fishkin and the Deliberative Opinion Poll: Lessons from a Study of the Granada 500 Television Program.” Political Communication 12 (April): 147– 56.Google Scholar
Druckman James N., and Kjersten R. Nelson. 2003. “Framing and Deliberation: How Citizens' Conversations Limit Elite Influence.” American Journal of Political Science 47 (October): 729– 45.Google Scholar
Elster Jon. 1998. Deliberative Democracy. Cambridge: Cambridge University Press.
Feldman Stanley. 1995. “Answering Survey Questions: The Measurement and Meaning of Public Opinion.” In Political Judgment: Structure and Process, ed. M. Lodge and K. McGraw. Ann Arbor: University of Michigan Press, 249– 70.
Fishkin James S. 1995. The Voice of the People. New Haven, CT: Yale University.
Fishkin James S., and Robert C. Luskin. 1999. “Bringing Deliberation to Democratic Dialogue.” In A Poll with a Human Face, ed. Maxwell McCombs and Amy Reynolds. Mahwah, NJ: Lawrence Erlbaum Associates, 3– 38.
Fraser Nancy. 1991. “Rethinking the Public Sphere: A Contribution to the Critique of Actually Existing Democracy.” In Habermas and the Public Sphere, ed. Craig J. Calhoun. Cambridge, MA: MIT Press, 109– 42.
Gastil John. 2000. By Popular Demand: Revitalizing Representative Democracy through Deliberative Elections. Berkeley: University of California Press.
Gastil John, and James P. Dillard. 1999. “Increasing Political Sophistication through Public Deliberation.” Political Communication 16 (February): 3– 23.Google Scholar
Gramlich Edward. 1998. Is It Time to Reform Social Security? Ann Arbor: University of Michigan Press.
Greene William H. 2000. Econometric Analysis. 4th ed. Upper Saddle River, NJ: Prentice Hall.
Greider William. 1992. Who Will Tell the People. New York: Touchstone.
Gutmann Amy, and Dennis Thompson. 1996. Democracy and Disagreement. Cambridge, MA: Belknap Press.
Habermas Jürgen. 1989. The Structural Transformation of the Public Sphere: An Inquiry into a Category of Bourgeois Society. Cambridge, MA: MIT Press.
Habermas Jürgen. 1997. Between Facts and Norms. Cambridge, MA: MIT Press.
Hardin Russell. 1999. “Deliberation: Method, Not Theory.” In Deliberative Politics: Essays on Democracy and Disagreement, ed. Stephen Macedo. New York: Oxford University Press, 103– 19.
Huckfeldt Robert, Paul E. Johnson, and John Sprague. 2004. Political Disagreement: The Survival of Diverse Opinions within Communication Networks. New York: Cambridge University Press.
Huckfeldt Robert, and John Sprague. 1995. Citizens, Politics, and Social Communication. Cambridge: Cambridge University Press.
Imai Kosuke. N.d. “Do Get-Out-The-Vote Calls Reduce Turnout? The Importance of Statistical Methods for Field Experiments.” American Political Science Review. Forthcoming.
Janis Irving L. [1972] 1982. Groupthink: Psychological Studies of Policy Decisions and Fiascoes. 2nd ed. New York: Houghton Mifflin.
Jones Bryan D. 2001. Politics and the Architecture of Choice. Chicago: University of Chicago.
King Gary, James Honaker, Anne Joseph, and Kenneth Scheve. 2001. “Analyzing Incomplete Political Science Data: An Alternative Algorithm for Multiple Imputation.” American Political Science Review 95 (March): 49– 70.Google Scholar
Kuklinski James H., Paul J. Quirk, Jennifer Jerit, David Schwieder, and Robert F. Rich. 2000. “Misinformation and the Currency of Citizenship.” Journal of Politics 62 (August): 790– 816.Google Scholar
Leamer Edward E. 1978. Specification Searches: Ad Hoc Inference with Nonexperimental Data. New York: John Wiley & Sons.
Levine Gary M., Jamin B. Halberstadt, and Robert L. Goldstone. 1996. “Reasoning and the Weighting of Attributes in Attitude Judgments.” Journal of Personality and Social Psychology 70 (February): 230– 40.Google Scholar
Manin Bernard. 1987. “On Legitimacy and Political Deliberation.” Political Theory 15 (August): 338– 68.Google Scholar
Mansbridge Jane. 1999. “Everyday Talk in the Deliberative System.” In Deliberative Politics: Essays on Democracy and Disagreement, ed. Stephen Macedo. New York: Oxford University Press, 211– 42.
McLeod Jack M., Dietram A. Scheufele, and Patricia Moy. 1999. “Community, Communication, and Participation: The Role of Mass Media and Interpersonal Discussion in Local Political Participation.” Political Communication 16 (July): 315– 36.Google Scholar
Mendelberg Tali, and John Oleske. 2000. “Race and Public Deliberation.” Political Communication 17 (April): 169– 91.Google Scholar
Mill John Stuart. [1859] 1947. On Liberty. New York: Appleton–Century–Crofts.
Mondak Jeffery J. 2000. “Reconsidering the Measurement of Political Knowledge.” Political Analysis 8 (Winter): 57– 82.Google Scholar
Mutz Diana C. 2002. “Cross-Cutting Social Networks: Testing Democratic Theory in Practice.” American Political Science Review 96 (March): 111– 26.Google Scholar
Mutz Diana, and Paul Martin. 2001. “Facilitating Communication across Lines of Political Difference: The Role of Mass Media.” American Political Science Review 95 (March): 97– 114.Google Scholar
Page Benjamin I. 1996. Who Deliberates? Mass Media in Modern Democracy. Chicago: University of Chicago Press.
Page Benjamin I., and Robert Y. Shapiro. 1999. “The Rational Public and Beyond” In Citizen Competence and Democratic Institutions, ed. Stephen L. Elkin and Karol Edward Soltan. University Park: Pennsylvania State University Press, 93– 115.
Petty Richard E., and Jon A. Krosnick. 1995. Attitude Strength: Antecedents and Consequences. Mahwah, NJ: Lawrence Erlbaum Associates.
Rosenbaum Paul R., and Donald B. Rubin. 1983. “The Central Role of the Propensity Score in Observational Studies for Causal Effects.” Biometrika 70 (April): 41– 55.Google Scholar
Sanders Lynn M. 1997. “Against Deliberation.” Political Theory 25 (June): 347– 77.Google Scholar
Shadish William R., Thomas D. Cook, and Donald T. Campbell. 2001. Experimental and Quasi-Experimental Designs for Generalized Causal Inference. Boston: Houghton Mifflin.
Sherman Robert P. 2000. “Tests of Certain Types of Ignorable Nonresponse in Surveys Subject to Item Nonresponse or Attrition.” American Journal of Political Science 44 (April): 362– 74.Google Scholar
Vinokur Amiram, and Eugene Burnstein. 1974. “The Effects of Partially Shared Persuasive Arguments on Group Induced Shifts: A Group Problem Solving Approach.” Journal of Personality and Social Psychology 29 (March): 305– 15.Google Scholar
Vinokur Amiram, and Eugene Burnstein. 1978. “Depolarization of Attitudes in Groups.” Journal of Personality and Social Psychology 36 (August): 872– 85.Google Scholar
Ward James D. 2004. Judicial Council of California Civil Jury Instructions. Albany, NY: LexisNexis Matthew Bender.
Wilson Thomas, D., and John Schooler. 1991. “Thinking too Much: Introspection Can Reduce the Quality of Preferences and Decisions.” Journal of Personality and Social Psychology 60 (February): 181– 92.Google Scholar
Young Iris Marion. 1996. “Communication and the Other: Beyond Deliberative Democracy.” In Democracy and Difference, ed. S. Benhabib. Princeton, NJ: Princeton University Press, 120– 35.
Zaller John. 1992. The Nature and Origins of Mass Opinion. Cambridge: Cambridge University Press.
Zaller John, and Stanley Feldman. 1992. “A Simple Theory of the Survey Response.” American Journal of Political Science 36 (August): 579– 616.Google Scholar
Figure 0

Survey Respondents by Treatment and Control Groups

Figure 1

The Effects of Deliberative Forum Participation on Correct, Incorrect, and Don't Know Respones to Social Security Knowledge Questions

Figure 2

The Effects of Deliberative Forum Participation on Three Social Security Reform Preferences

Figure 3

The Effects of Deliberative Forum Participation by Strength of Prior Opinion

Figure 4

The Effects of Discussion on Social Security Reform Preferences by Strength of Opinion

Submit a response

Comments

No Comments have been published for this article.