A common assumption in political economics is that individuals can evaluate the implications of alternative policies and that they know which policies are in their best interest. However, individuals are rarely fully informed about proposed and existing policies, and a growing body of evidence suggests that individuals are not fully informed about the public policies relevant for their choices (Chetty et al., Reference Chetty, Looney and Kroft2009; Chetty and Saez, Reference Chetty and Saez2013). Knowledge about the pension system have been found to be particularly low (Chan and Stevens, Reference Chan and Stevens2008; Boeri and Tabellini, Reference Boeri and Tabellini2012), which is especially problematic in times where the pension system is being reformed and the future individual pension wealth depends increasingly on individual decisions. If individuals do not understand the changes in incentives, then they may make decisions that they would not have made had they been fully informed about their consequences (Mastrobuoni, Reference Mastrobuoni2011; Chetty and Saez, Reference Chetty and Saez2013). Public authorities typically attempt to reduce the detrimental impact of low information through public information campaigns (Henry and Gordon, Reference Henry and Gordon2003). The aim of the present paper is to investigate if the short-term effects, as documented in a previous paper (Finseraas and Jakobsson, Reference Finseraas and Jakobsson2014a) of public information materials on knowledge among the general public persists in the longer run.
Rational ignorance refers to how individuals lack incentives to base their voting decisions on accurate information (Downs, Reference Downs1957). Given that each individual has a negligible probability of affecting the outcome in an election, individuals will not have an incentive to acquire information before voting. But also when information is cheap and the benefits of knowledge are more imminent, people may lack knowledge about the functioning and rules in public policy. One reason may be lack of cognitive ability; another may be that people behave myopic. Finseraas and Jakobsson (Reference Finseraas and Jakobsson2014a, Reference Finseraas and Jakobsson2014b) show that there is a short-term knowledge effect from the information intervention studied in this paper. In the present paper, we investigate how the short-term effect changes in the medium-run. The short-term effect may persist over time, implying that the medium-term knowledge effect is the same as the short-term effect. Another possibility is that the short-term effect erodes over time, a possibility that is well in line with the fact that people tend to forget (Rubin and Wenzel, Reference Rubin and Wenzel1996), and the notion that information campaigns tend to fail (see the seminal paper by Hyman and Sheatsley, Reference Hyman and Sheatsley1947). But there is of course also a possibility that knowledge due to the information has increased even more in the medium-term than in the short-term. If the information spurs an interest among the informed individuals they may acquire more information than they otherwise would have. Thus, the question is: will increased knowledge about the functioning of social security systems persist over time, decrease over time, or increase even further over time?
The experiment
To study the effect of public information on knowledge we sent a survey questionnaire to 3,000 individuals between 40 and 67 years of age. A total of 1,500 of these individuals were randomly allocated to a treatment group and 1,500 to a control group. The treatment consisted of receiving a standard information brochure from the Norwegian Labour and Welfare Service (NAV) on the recently implemented changes in the pension system. The brochure was emailed to the respondents in the treatment group as a link to a web page with a pdf (Portable Document Format) of the brochure six days before they received the first survey. Both groups received the survey questionnaire at the same time. The brochure contains general information about how labor market behavior affects future individual pension accounts, and about the possibilities to combine retirement and work. We give a short description of its content as well as a web-link to the brochure in Appendix A.
The first survey (wave 1) was conducted September 10–24, 2012. A second wave (wave 2) of the survey was sent to everyone who responded to the first survey, and the second round was conducted February 6–20, 2013. The surveys were administered by Yougov (www.yougov.no). The respondents were recruited from Yougov's panel of individuals who have volunteered to participate in surveys. These individuals get the opportunity to participate in web-based surveys on different themes provided regularly by Yougov. Participation in the panel and in each survey is voluntary and respondents earn credits from answering surveys; these credits can be used for participating in lotteries, as payment for goods, or as charity.
It should be stated that the brochure was part of a larger information campaign on the reform, a campaign which included ads in newspapers, radio, and TV, as well as cooperation with main news desks to ensure accurate reporting of the implemented changes. At the end of 2010 – two years before our survey experiment – the brochure was distributed to everyone aged 62 or above after January 1, 2011, thus, for those aged 63 or above in our sample the treatment is a reminder rather than new information.
Data and descriptive statistics
A total of 662 individuals from the treatment group answered both rounds of the survey (44%), while 736 individuals from the control group answered (49%). This 5% points difference in response rate is statistically different from zero according to a two sample z-test to compare sample proportions (z-value = −2.7). The differential response rate could be due to the different efforts required by the treatment and control groups. Respondents in the treatment group both received the brochure and were asked to answer the questionnaire, while those in the control group were only asked to answer the questionnaire. It is also possible that members in the treatment group thought they should know the answers because they received the brochure, and that those who are unsure about their answers decide to not respond rather than to show their ignorance. Since the focus is the difference between the short- and medium-term effects of the treatment, the unbalanced sample is not of the same concern as in a standard experiment.
Table 1 provides descriptive statistics of the treatment and control group, respectively. Comparing the treatment and control group we see that the treatment group consists of a higher share of men, has slightly lower level of education and income. However, we do not believe that these differences are a substantial concern for the primary aim in this paper, i.e., whether or not there is a difference between the short- and medium-term effects of a public information campaign on knowledge. We compare potential differences in the treatment effect over the two measurement periods, and the small but in some regards significant unbalance between treatment and control group is equal in both waves. Thus, a potential bias in wave 1 should follow in wave 2, making the comparison between waves unbiased. With regards to generalizability to the general population, our sample includes more individuals with university education than does the population between 40 and 67 years of age (52.7% compared with 31.2%). The sample also includes more men (59.8% compared with 50.9%). Since there were a clear drop in responses between the waves; from a total response rate of 77% in the first wave to 46% in the second wave attrition may be a problem. The share with university education is stable between the waves (from 53.2% to 52.7%), while there is an increase in the share of men (from 54.4% to 59.8%). In the present paper, we report results including individuals that participated in both waves so our findings will not depend on changes of composition between the waves; however, the generalizability of the results are weaker as compared to a sample that is more representative of the total population.
Descriptive statistics, mean values
Note: Statistically different from the control group at: ***p < 0.01, **p < 0.05, *p < 0.1
Results
In this section, we show the Intention-to-treat effects (ITT). With randomized treatment the ITT is the differences in mean of the outcome between the treatment and control group. In ITT estimates all individuals receiving the treatment are included as treated even if they were compliers or not, and this is the preferred estimate in order to evaluate the campaign in itself. Given the unbalanced sample between treatment and control we include a set of individual covariates in the regressions.Footnote 1 Since the focus of the paper is the potential difference between the short- and medium-term effects of the treatment, the unbalanced sample is not of the same concern as in a standard experiment. The concern in our situation regards the possibility to extrapolate out of sample.
Before moving on to the regression results we report the raw means for the treatment and control groups in the first and second waves (Table 2). The results from wave 1 show that individuals who were provided with the information are significantly more likely to state that the pension reform succeeded in simplifying the pension system (8 percentage points more likely). The treated group is also 8 percentage points more likely to correctly identify that the new pension system will imply a higher pension if you decide to retire later. The difference in correctly identifying that an increasing life expectancy will reduce pensions, and correctly stating that an increasing unemployment in your own cohort will not reduce your pension were 3 percentage points in both cases. The differences in the second wave are much smaller and for the first and last questions, the control group actually reports higher numbers. The questions used to construct the dependent variables are reported in Appendix B.
Mean answer to the dependent variables for the treatment and control groups in first and second waves
Note: Treatment, 1 if respondent was allocated to the group who received the brochure, 0 otherwise; Easier, 1 if the respondent states that the pension has become easier (to a great or moderate extent), 0 otherwise; Retire later, 1 if the respondent correctly states that her monthly pension payments will be higher if she decides to retire later in life, 0 otherwise; Life expectancy, 1 if the respondent correctly states that her pension will be lower if the life expectancy of her cohort increases, 0 otherwise; Unaffected unemployment, 1 if the respondent correctly states that her pension is unaffected by increases in the unemployment rate of her age group, 0 otherwise.
Table 3 shows the marginal effects from a probit regression on the four different outcome variables at waves 1 and 2, respectively. The estimates for wave 1 show that individuals who were provided with the information are significantly more likely to state that the pension reform succeeded in simplifying the pension system (8 percentage points more likely). The treated group is also 10 percentage points more likely to correctly identify that the new pension system will imply a higher pension if you decide to retire later and 7 percentage points more likely to correctly state that an increasing unemployment in your own cohort will not reduce your pension. However, we cannot reject the null hypothesis of no difference between the treated and control group in correctly identifying that an increasing life expectancy will reduce pensions. As reported above there are some, but only small differences, between the raw means in Table 2 and the regression results in Table 3.
Intention to treat estimates: short- and long-term effects, marginal effects from probit regression
Note: ***p < 0.01, **p < 0.05, *p < 0.1, standard errors in parenthesis.
Treatment, 1 if respondent was allocated to the group who received the brochure, 0 otherwise; Easier, 1 if the respondent states that the pension has become easier, 0 otherwise; Retire later, 1 if the respondent correctly states that her monthly pension payments will be higher if she decides to retire later in life, 0 otherwise; Life expectancy, 1 if the respondent correctly states that her pension will be lower if the life expectancy of her cohort increases, 0 otherwise; Unaffected unemployment, 1 if the respondent correctly states that her pension is unaffected by increases in the unemployment rate of her age group, 0 otherwise. All eight regressions include individual covariates, these results also displaying the control variables can be found in Table A1 in Appendix C. The change between waves 1 and 2 is statistically significant for the three outcome variables Easier, Retire Late, Unaffected by Unemployment at the 1%, 5% and 10% level, respectively.
Moving on to the results from wave 2 in Table 3 we see that none of the estimates are statistically significantly different from zero, i.e., we cannot reject the hypothesis that the treatment has no effect on the knowledge of the pension system. The interpretation of the results is hence that the positive knowledge effect of the treatment seen for three out of four outcome variables in wave 1 shows indication to have disappeared four months later in wave 2. Finally, we have also (for each of the four outcome variables) tested if the change in the knowledge between waves 1 and 2 between the treated and control group is statistically significant within each of the outcome variables. The change between waves 1 and 2 is statistically significant for the three outcome variables Easier, Retire Late, Unaffected by Unemployment at the 1%, 5% and 10% level, respectively.
Conclusion
We find that respondents who were allocated to a group that receives an information brochure about the pension system are more likely to answer basic questions on the pension system correctly a week later. However, our main contribution is the finding that, when the same questions are asked four months later the effect of the information brochure has disappeared completely. The findings illustrate the limits of public information campaigns to improve knowledge about an important public policy reform, in this case represented by a pension reform.
Appendix A
The information brochure describes the new Norwegian pension system, including the possibility of retiring from 62 years of age, and of combining work and pension, that the yearly pension payments will be higher if you decide to retire later, and that the yearly pension payments will decrease as life expectancy increases. Furthermore, it gives examples of how the pension payment depends on life expectancy, how long you need to postpone retirement to compensate for an increase in life expectancy, and how your yearly pension payment depends on retirement age. The reader is also directed to a web page and a telephone number where she can get additional information on the pension rules and calculate her yearly pension payments. The brochure is posted online on this web page: https://niklasjakobsson.files.wordpress.com/2010/06/brosjyre.pdf
Appendix B
One of the main goals of the pension reform was to achieve an easier pension system. To what extent do you think the reform has been successful with this?
To a great extent
To a moderate extent
To a little extent
To a limited extent
What do you think will happen with your pension if you choose to postpone your retirement?
The monthly pension payment will be lower
It will not have an effect on the monthly pension payment
The monthly pension payment will be higher
What do you think will happen with your pension if life expectancy in your age cohort increases?
The monthly pension payment will be lower
It will not have an effect on the monthly pension payment
The monthly pension payment will be higher
What do you think will happen if the unemployment rate in your age cohort increases?
The monthly pension payment will be lower
It will not have an effect on the monthly pension payment
The monthly pension payment will be higher
