Public opinion, policy feedback and programme effectiveness

There is no shortage of research into welfare and welfare reform, including studies on the effect welfare reform has had on a wide array of social and economic factors, including welfare caseloads, labour force participation, the self-sufficiency and wellbeing of welfare recipients and leavers, family formation behaviours, and children's wellbeing and cognitive development (for reviews of the literature, see Blank, Reference Blank2002, Reference Blank2006; Sawhill et al., Reference Sawhill, Weaver, Haskins and Kane2002). Fewer studies have analysed whether or not welfare policy reform has changed the extent to which the general public supports or opposes current governmental welfare policies or anti-poverty policies and spending. Given the importance of public opinion on government policy, any shift in public opinion would be an important outcome.

A number of scholars interested in policy feedback theory have examined the potential of policy change to influence citizen action and public opinion. Pierson (Reference Pierson1993) outlined a framework for analysing the effect of policy on mass politics and specifically addressed the possibility that the interpretive effects of policy can change public opinion. Despite this call for analysis, there has been little empirical research done directly on policy feedback and the factors that influence it.

Existing research on government performance, while not directly addressing the issue of interpretive effects in policy feedback, supports the idea. Fiorina (Reference Fiorina1981) theorised that voters judge political actors and institutions based on their past performance, and then either reward or punish them at the polls. Examining the possibility that the public can learn from policy outputs, Gusmano et al. (Reference Gusmano, Schlesinger and Thomas2002) argue that when individuals have experience with a policy, in their case employees' experience with employer health care policies, policy learning is a likely occurrence. A common element in these research models is the assumption that policy learning is possible because the public is knowledgeable about a policy and has opinions on the policy's performance, effectiveness or applicability. Mettler and Soss (Reference Mettler and Soss2004) explain this aspect of policy feedback as policy's role in framing agendas, problems and evaluations and thus directing the public's perception of critical policy issues.

Following this line of evidence, personal experience with the welfare system, as either a client or a case worker, should have some direct effect on an individual's opinion of the programme and broader social spending policies. Both quantitative and qualitative research on welfare reform provide interesting data on this question and lead to critical policy implications. Research on case worker perceptions indicate that although many support the goals of welfare reform, their experience with the system leads them to cite a number of structural barriers to programme success, including high caseloads and imperfect inter-agency coordination (Bell, Reference Bell2005). Comparing social worker and client perspectives, Bullock (Reference Bullock2004) found that welfare recipients supported increased funding and more progressive policies than social workers did. In-depth interview data from the Three City Study indicate that recipients view reforms with ‘cautious optimism’, but are also in favour of more child care, and educational and training assistance (Burton et al., Reference Burton, Cherlin, Francis, Jarrett, Quane, Williams and Cook1998). Examining civic engagement as a critical outcome, Soss (Reference Soss1999) concludes that, through their direct exposure to the system, welfare recipients develop opinions that affect their broader political actions and participation. This existing research clearly demonstrates that welfare reform has had some impact on the opinions of both case workers and recipients. The question remaining, however, is whether those with no direct experience with the policy – the general public – are also affected by their perception of the programme's success or failure. This is a vital question for the theory of policy feedback, which looks for a broad-based public reaction to policy change.

Many scholars have addressed the question of where public preferences on government aid to the poor originate, but they do so almost uniformly from a political orientation angle rather than a policy one. Such research attributes public support of poverty spending to the demographics, ideology and perceptions of the American public. Models commonly centre around individual political factors such as ideology and party identification, as well as including demographic factors such as race, age and gender (Cook and Barrett, Reference Cook and Barrett1992; Gilens, Reference Gilens2001; Kluegel and Smith, Reference Kluegel and Smith1986; Swank, Reference Swank2005). For example, Jacoby (Reference Jacoby1994) finds that individuals' party affiliations and ideological orientations consistently affect their preferences on government spending in a variety of programmes areas; and Eismeier (Reference Eismeier1982) points out that individual partisan attachments play a central role in determining preferences regarding welfare spending. Other identified factors important in determining welfare spending preferences are individual perceptions of the economy (Stevenson, Reference Stevenson2001), of government (Eismeier, Reference Eismeier1982) or of welfare recipients themselves (Cook and Barrett, Reference Cook and Barrett1992).

Despite the traditional focus on individual political variables, some previous spending research does lend support to the belief that individual perceptions of the success of policy matter to preferences regarding the policy and spending on it. Using pre-reform data, Cook and Barrett (Reference Cook and Barrett1992) show that opinions of programme effectiveness are a critical determinant of public support for welfare programmes. Studies of public attitudes toward other types of government spending or government spending in general demonstrate that policy changes do impact on public spending preferences. Wlezien (Reference Wlezien1995) demonstrates how the public acts as a thermostat for the government's spending policies. When government changes policies or spending levels, the public reacts to it. If the public was previously dissatisfied with spending levels, they may react positively to government actions that address this dissatisfaction, resulting in a readjustment of public opinion, and potential satisfaction with the new status quo.

Empirical research on public opinion and welfare spending, however, has not produced strong findings to back up these theories and has largely either ignored or discounted the related issue of programme effectiveness. In examining the shift in public opinion in the 1990s, Schneider and Jacoby (Reference Schneider and Jacoby2005) conclude that the influence of elite messages through the media played a large role in temporarily reducing support for welfare spending in the mid-1990s, when public opinion reached its lowest point.

Their findings raise a potential difficulty in attributing any evidence of opinion change to individual perceptions of the programme rather than just the influence of media reports or elite messages covering it. At the same time, however, Schneider and Jacoby (Reference Schneider and Jacoby2005) may also be oversimplifying opinions and their sources. Other research (for example, Weatherford, Reference Weatherford1983; Nadeau et al., Reference Nadeau, Niemi and Amato2000) demonstrates that policy opinions are not entirely predictable, nor are they merely parroted messages from the media. Rather, individuals apply their own subjective interpretations to factual reports they receive through the media, rather than uniformly taking an objective or factual stance. Part of their subjectivity stems directly from their own personal experiences, stories they have heard elsewhere and general gut reactions (Weatherford, Reference Weatherford1983).Footnote ¹ As a result, individuals react differently to reports such as those on the ‘success’ of welfare reform, making the potential linkages between their subjective judgements of the programme and their views on programme spending an important policy question.

Recent research begins to assess the possible power of the 1996 welfare reforms as a policy feedback mechanism. On the one hand, some scholars offer evidence that the 1996 reforms changed the direction of falling public support of welfare programmes to more moderate levels (for example, Shaw and Shapiro, Reference Shaw and Shapiro2002). More specifically, research has shown that the enactment of the reforms has softened the public's perception of welfare recipients (Hetling et al., Reference Hetling, McDermott and Mapps2008). On the other hand, Soss and Schram (Reference Soss and Schram2007) use aggregate-level data and conclude that policy feedback did not work in the case of welfare and, instead, argue that the case may be used to further our understanding of the circumstances under which policy feedback can fail. They theorise that although the 1996 reforms were a very visible policy issue, because the general public has only a distant relationship with the programmes, opinion changes are less likely.

By investigating individual perceptions of programme effectiveness and the reasons particular opinions are formed, our research takes a new approach to assessing the impact of welfare policy change on opinion change, and we are able to investigate the opinion effects of welfare reform on a new, more personal level. We hypothesise that the existing research's focus on aggregate trends hides an important mechanism upon which the success of policy feedback may depend. We contend that public support of policy initiatives and government spending depends on individual awareness and perceptions of programme effectiveness. By examining individual-level data, we acknowledge the possibility that individual perceptions may change, even when aggregate trends do not change or change very little. If an individual-level connection exists between perceptions of policy effectiveness and spending preferences, then there is evidence of policy feedback even if changes do not occur in the mass public. Perhaps the 1996 reforms did not lead to mass feedback effects because advocates or politicians did not seize the opportunity, not because the potential did not exist.

We hypothesise that welfare reform has affected preferences on poverty spending through individuals' perceptions of its effectiveness. Specifically, those citizens who view the reforms as effective should be relatively content with the new status quo: reflected in a satisfaction with current government spending levels. Those who view the reforms as ineffective, on the other hand, should be dissatisfied with spending levels. Further, the reasons behind their disapproval of the reforms' outcomes should determine the direction of their spending dissatisfaction. Those who believe the reforms are ineffective because they went too far in setting up requirements in order to receive assistance should feel spending is too low, while those who do not think the reforms went far enough should find spending levels too high.

Method and analysis

To test our hypotheses that opinions of welfare reform and the reasoning behind those opinions affect support for spending on the poor, we use a survey of public welfare attitudes sponsored by the Kaiser Family Foundation, National Public Radio and Harvard University's Kennedy School of Government. The Poverty in America survey was a telephone survey conducted in English and Spanish from 4 January to 27 February 2001 among a nationally representative, random sample of 1,942 American adults.Footnote ² This survey is very useful for our purposes because it contains questions not only about awareness of welfare reform, but also about perceptions of the effectiveness of the reforms and the reasons behind these perceptions.

Unlike most existing analyses of spending preferences, we are not asserting a linear relationship in our hypothesis. Whereas attitudes such as party affiliation and ideology should have linear connections to spending attitudes – as respondents become more conservative or Republican, preferences for spending cuts increase – we predict a categorical relationship between perceived effectiveness and spending preferences. As a result, we use multinomial logistic regression as our analysis method.

The dependent variable in our analysis is citizens' attitudes toward government spending on aid to the poor.Footnote ³ The question asks respondents whether they feel government is spending too much, too little or about the right amount on assistance to poor people. The broad nature of the question is an advantage for our research design. Had the question focused on welfare spending specifically, we would have faced a clear simultaneity issue in that perceptions of the welfare programme, whether related to spending or to effectiveness, are likely related and not necessarily in just one direction. The possibility that one's feelings on welfare spending affect feelings on programme effectiveness is as easily argued as the reverse. This is less likely to be the case when thinking about general spending on the poor and specific welfare effectiveness.

Our key independent variables are taken from a series of questions about welfare reform. The first question asks respondents whether they know if there was significant welfare reform in the past five years. Those who knew welfare reform had occurred were then asked whether they felt the reforms were working well or not working well. Next, each of these groups – those who thought it was working well and those that thought it was not – were asked the primary reason for their judgements. Those who thought it was not working well were asked which of two reasons – that the reforms were denying too many people help or that the reforms had not cut the welfare rolls enough – came closer to the reason for their judgement.

These questions provide us with the independent variables needed to test our hypotheses. First, we can establish whether perceptions of effectiveness influence attitudes toward spending for the poor. Second, we can test whether the reasons behind citizens' perceptions of ineffectiveness drive their spending preferences.

Hypothesis test 1 – perceptions of effectiveness

Our first hypothesis is that those Americans who think the welfare reforms have worked are more likely to be content with current levels of spending than they are to want spending levels increased or decreased. We also hypothesise that this group is more likely to be content with current levels than are other groups. Table 1 provides a cross-tabulation of perceptions of welfare and spending attitudes. The columns represent those who do not know welfare reform occurred (our baseline group),Footnote ⁴ those who are cognisant that reform took place and think the welfare programme works well as a result, and those who know reforms took place but believe the system does not work well.

TABLE 1.

Relationship between perceptions of welfare and spending attitudes

Notes: ^aSignificantly different from ‘System works worse’ at p ≤ 0.01 (chi-square).

^bSignificantly different from ‘Didn't know reform occurred’ at p ≤ 0.01 (chi-square).

All significance tests are two-tailed.

The table demonstrates initial support for the hypothesis. Those who believe the reforms have worked well are more likely to think spending levels are about right than to feel they are either too high or too low: 46 per cent versus 19 per cent and 35 per cent respectively. In addition, more of these citizens are content with spending levels than are either those who did not know reform occurred or those who believe the reforms are ineffective. Forty-six per cent of those perceiving the reforms as effective believe current spending levels are fine, versus 41 per cent of those with no knowledge of reform and 18 per cent of those believing the reforms to be ineffective (a significant difference).

These results lend direct support to our first hypothesis: that those satisfied with reforms will also be satisfied with current spending levels. In order to provide a more stringent test, however, we need to control for other potential influences on spending preferences. To do so we construct a multinomial logistic regression model including our test variables for the first hypothesis, as well as control variables for the potential factors others demonstrate influence attitudes toward spending on welfare and the poor. Attitudinal controls include citizens' perceptions of whether jobs are available to those on welfare, and whether available jobs pay enough for families on welfare to subsist (each coded 1 for those who believe jobs are available and that the jobs pay enough, 0 for those who do not). Demographic controls include gender, education, income, age, whether the respondent is currently receiving welfare benefits and whether the respondent is African-American.

We also include political controls in the model. To control for the effects of party affiliation, we include two dummy variables, one for respondents who identify themselves as Republicans and one for Democrats. Ideological orientations are also controlled for using two dummy variables. For the first variable, a value of 1 was set for those who identify as conservative; for the second, those who identify as liberals took on a value of 1. Moderates are captured by the constant. These political controls are especially important, given previous findings that ideology is central to welfare spending preferences (Jacoby, Reference Jacoby1994).

The multinomial model measures the log likelihood of respondents giving one answer as opposed to another. In this model, the comparison answer is spending the right amount. According to our hypothesis, respondents who believe reforms have worked well should be significantly more likely to answer that spending levels are right than they are to answer that spending levels are either too low or too high. This predicts negative and significant coefficients for the effectiveness variable in both the spending too much and spending too little categories. Such a result would demonstrate that these respondents are significantly less likely to feel spending levels need either increasing or decreasing than to feel spending is about right.

Table 2 contains the predicted probabilities for the answer categories based on perceptions of reform effectiveness. (Full results for the model estimation are included in the Appendix Table.) The predicted probabilities demonstrate very similar results to the simple crosstabular results in Table 1, supporting the hypothesis that perceptions of reform effectiveness drive contentment with welfare spending levels. Our specific hypothesis is that those who believe the system works better as a result of reforms will be more likely to express satisfaction with spending levels than to want either higher or lower levels; and that they will also be more satisfied with spending levels than will those who either do not know reform occurred or who believe the reforms have worsened the programme.

TABLE 2.

Effects of welfare perceptions on spending attitudes – predicted probabilities

Notes: ^aSignificantly different from ‘System works worse’ at p ≤ 0.01 (chi-square).

^bSignificantly different from ‘Didn't know reform occurred’ at p ≤ 0.01 (chi-square).

¹Significantly different from ‘spending right amount;’ p ≤ 0.05 (multinomial A-2).

Table entries are predicted probabilities of spending positions based on perceptions of programme effectiveness calculated from the full model in the appendix table, with other variables held constant at values representing a politically independent, ideologically moderate, white female, of average education, age and income and who does not receive welfare. All significance tests are two-tailed.

Respondents who believe that reforms have made the welfare system better are significantly more likely to feel that spending levels are correct than to prefer either of the other options. Specifically, those who believe reforms have worked have a 54 per cent probability of expressing satisfaction with spending levels. In contrast, they have only 32 per cent and 14 per cent probabilities of feeling the government is spending too little or too much, respectively. These results directly support the first expectation of our hypothesis.

Those who perceive welfare reform has made the system work worse and those not aware that reform occurred follow quite different answer patterns than that of the effective group. Citizens who believe the reforms have been ineffective are far more likely to say that spending on the poor is too little rather than too much or adequate. Those who have not heard about reform are more evenly dispersed; they have a 41 per cent probability of satisfaction with spending levels and slightly lower probabilities for the other two categories. Overall the spending preferences for those who feel the system works better are significantly different from the preferences of either of the other groups. This finding also supports our hypothesis.

As stated previously, these results demonstrate that for those who feel that welfare reforms have been ineffective or that the programme works poorly, the most likely attitude is that spending levels are too low. They are significantly more likely to feel this way than to feel spending levels are about right, and they are significantly more likely to feel this way than are citizens who do not feel the system benefited from reform or who do not know reforms occurred. The question is why.

Our second hypothesis deals with these feelings that reforms have been ineffective. But rather than arguing for a general effect as we did here in our first hypothesis regarding effectiveness, we expect that the reasons individuals find the reforms ineffective should influence their subsequent spending preferences. We now turn to a test of this second hypothesis.

Hypothesis test 2 – perceptions of ineffectiveness

The reasons behind people's perceived views of reform ineffectiveness are likely to drive their attitudes toward spending levels. Specifically, two primary reasons for perceptions of ineffectiveness are that welfare reform went too far and the programme is denying too many people assistance, and that it did not go far enough: reasons that would logically lead to different spending beliefs. For those who think welfare reform went too far, and is thereby not doing enough to help the poor, it seems reasonable to expect that they would find spending levels too low. On the other hand, those who feel that the reforms were ineffective because they were not strict enough should be most likely to express that spending levels are too high. Here again we would expect that these preference effects will exist not only within each group but also among the groups.

We use two new variables to test this hypothesis. The survey includes, as described above, a question about reasons for dissatisfaction with welfare reform with the two response choices: the new law denies too many people assistance or the new law did not cut the rolls enough. Table 3 presents a crosstabular analysis of attitudes toward spending as related to perceptions of reform ineffectiveness and reasons for them. The results demonstrate substantial support for the second hypothesis.

TABLE 3.

Relationship between reasons for welfare perceptions and spending attitudes

Notes: ^aSignificantly different from ‘Worse – denying help’ at p ≤ 0.01 (chi-square).

^bSignificantly different from ‘Worse – not cut enough’ at p ≤ 0.01 (chi-square).

^cSignificantly different from ‘Didn't know reform occurred’ at p ≤ 0.01 (chi-square).

All significance tests are two-tailed.

Americans who feel welfare reform has made the programme worse because it cut too many poor people from the rolls are far more likely to perceive government aid levels as too low. Three-quarters of these respondents feel spending is too low, while only 18 per cent feel it is about right and 6 per cent find it too high. Not only are these citizens overwhelmingly more likely to find spending levels too low, these spending preferences are significantly different from those of every other group, as expected.

Dissimilarly, as hypothesised, those who feel reform did not go far enough in cutting the welfare rolls are comparatively more dissatisfied with spending levels, and a plurality, 46 per cent, believe the government is spending too much, as expected. Unexpectedly, however, a substantial 40 per cent feel spending levels are too low, fewer than believe levels are too high, but surprising nonetheless. These findings are consistent with the hypothesis, as is the pattern among groups. Those who feel that reforms did not cut enough individuals from the rolls have spending preferences that are significantly different from each of the other groups.

We include each of these answer options in the multinomial spending model as dummy variables, in place of the ineffectiveness variable, to provide a more controlled test of the hypothesis. If our hypothesis is correct, those respondents who perceive that welfare reform denies assistance to needy people should be most likely to feel spending levels are too low. Thus, we should find that these respondents are more likely (the highest probability) to say that spending levels are too low than they are to provide any other answer or than any other group is to give the same answer. For those respondents who perceive ineffectiveness because reforms did not cut enough people from the rolls, the probability of feeling that spending on the poor is too high should be significantly increased over other options and other groups.

Table 4 contains the probability transformations of the full multinomial model (also in the Appendix Table). Here again, the results for those who feel welfare reforms went too far in cutting the programme strongly support our hypothesis. First, this group has a 62 per cent chance of feeling that the government is currently spending too little on assistance to the poor. In contrast, they have only a 12 per cent probability of feeling spending is too high and a 26 per cent probability of satisfaction with spending levels, differences that are statistically reliable. Second, as in the crosstabs, this group's probabilities of answering this way are significantly different from each of the other groups' response probabilities.

TABLE 4.

Effects of reasons for welfare perceptions on spending attitudes – predicted probabilities

Notes: ^aSignificantly different from ‘Worse – denying help’ at p ≤ 0.01 (chi-square).

^bSignificantly different from ‘Worse – not cut enough’ at p ≤ 0.01 (chi-square).

^cSignificantly different from ‘Didn't know reform occurred’ at p ≤ 0.01 (chi-square).

¹Significantly different from ‘spending right amount;’ p ≤ 0.05 (multinomial A-2).

Table entries are predicted probabilities of spending positions based on perceptions of programme effectiveness calculated from the full model in the appendix table, with other variables held constant at values representing a politically independent, ideologically moderate, white female, of average education, age and income and who does not receive welfare. All significance tests are two-tailed.

Also similar to the crosstabular results, the multinomial model predicts mixed results among those who feel welfare reform did not cut enough people from the rolls. Although this group is the most likely of all of the groups to feel that spending on the poor is too high, as expected, they are still more likely (59 per cent probability) to feel spending levels are too low, contrary to the hypothesis. They are, however, significantly more likely to find spending levels too high than are either those who find the reforms effective or those who feel they went too far.