2 The pension reform

Norway and most other developed countries are facing a rapid increase in the proportion of elderly in the population due to increased longevity and lower birth rates. An increase in the number of pensioners relative to employees challenges the sustainability of pay-as-you-go pension systems. As a response to this demographic change, the government appointed in 2001 a Pension Commission that included representatives from all parliamentary parties, and in 2009, a broad majority in the Norwegian parliament voted in favor of a structural pension reform.

Under the new public pension system, individuals will each year earn pension rights equivalent to 18.1% of annual earnings up to a ceiling at about 120% of the average wage (see Christensen et al. (Reference Christensen, Fredriksen, Lien, Stølen, Holzmann, Palmer and Robalino2012) for an overview of the new pension system). The pension entitlements are placed in a notional account, which is adjusted annually by the wage growth. Upon retirement, the annual pension is determined by dividing the accumulated pension wealth with a divisor that mainly reflects expected number of years as a pensioner.Footnote ¹ Pension expenditure will still be financed by the pay-as-you-go-method.Footnote ² Another notable change is the abolishment of a fixed retirement age, previously at 67, and the introduction of a flexible retirement age, beginning at 62.

The reform provides strong incentives to postpone retirement. First, in the old system early retirement at 62 was heavily subsidized through the Early Retirement Scheme (Avtalefestet pensjon, AFP). This subsidy has now been removed for those in the private sector who had access to the AFP under the old system, while it is still in place for those in the public sector. Second, as the divisor implies that the amount of pension is tied to life expectancy, one's pension will be reduced if one does not postpone retirement as a response to an increase in the life expectancy of one's cohort. Third, the relationship between your earnings and amount of pension is strengthened and will now depend on one's work history throughout life.

The strengthened incentives to postpone retirement leads us to expect a positive effect of the reform on planned labor supply, conditional on knowledge of the pension system. However, the introduction of the flexible retirement age makes it easier to retire at a younger age. In isolation, the effect of more flexibility might lower labor supply, and thereby reduce the total positive effect of the reform on labor supply.

Knowledge and information about the incentives of the pension system are not randomly distributed, and very few studies have considered the relationship between knowledge and retirement plans from a causal angle. Duflo and Saez (2003) find that employees who were randomized into receiving information on a retirement savings plan were substantively more likely to join a savings program than those in the control group. Mastrobuoni (Reference Mastrobuoni2011), utilizes the introduction of the Social Security Statement to get exogenous variation in information on workers’ future pension, but finds no change in retirement behavior. He speculates that the information provided by the Social Security Statement might not be extensive enough to change behavior, an interpretation supported by others (see Liebman and Luttmer, Reference Liebman and Luttmer2011).

We conduct a survey experiment where we randomize information about the recently implemented reform to get exogenous information about the incentives. Then we study whether the information (i) improves knowledge about the pension system and (ii) increases planned labor supply when the respondents reach an age where they are allowed to retire. We describe the experiment in the next section.

3 Experimental setup and data description

To study the causal relationship between knowledge of the pension system and retirement plans, we sent a survey questionnaire to 3,000 individuals between 40 and 67 years of age. 1,500 of these individuals were randomly allocated to a ‘treatment group’ and 1,500 to a ‘control group’. The survey was conducted September 10–24, 2012, and was administered by Yougov (www.yougov.no). The respondents were recruited from Yougov's panel of individuals who have volunteered to participate in surveys. These individuals’ get the opportunity to participate in web based surveys on different themes provided regularly by Yougov. Participation in the panel and in each survey is voluntary and respondents earn credits from answering surveys, these credits can be used forparticipating in lotteries, as payment for goods, or as charity.

The treatment consisted of receiving a standard information brochure from the Norwegian Labour and Welfare Service (NAV) on the recently implemented changes in the pension system. The brochure contains general information about how labor market behavior affects future individual pension accounts, and about the possibilities to combine retirement and work. We give a short description of its content as well as a web-link to the brochure in the Appendix. The brochure was part of a larger information campaignon the reform, a campaign which included ads in newspapers, radio, and TV, as well as cooperation with main news desks to ensure accurate reporting of the implemented changes. At the end of 2010–2 years before our survey experiment – the brochure was distributed to everyone with the right to a pension after January 1, 2011, thus, for those in this age group the treatment is a reminder rather than new information. We examine this difference in treatment below.

The brochure we use should be considered a weaker information treatment than in Mastrobuoni (Reference Mastrobuoni2011), since ours does not contain individual information on estimated monthly pension payments at different ages of retirement (although it does contain information on how to obtain this information). Thus, the information might not be specific and extensive enough to have an impact on retirement planning.

The brochure was emailed to the respondents in the treatment group as a link to a web page with a pdf (Portable Document Format) of the brochure 6 days before they received the survey. Both groups received the survey questions at the same time. The web link to the brochure was inactive when they received the survey.Footnote ³ A total of 1,050 individuals from the treatment group answered the survey (70%), while 1,252 individuals from the control group answered (83%). This 13% points difference in response rate is striking and clearly statistically different from zero (t-value=8.81). The differential response rate could be due to the different efforts required by the treatment and control groups. Respondents in the treatment group both received the brochure and were asked to answer the questionnaire, while those in the control group were only asked to answer the questionnaire. It is also possible that members in the treatment group thought they should know the answers because they received the brochure, and that those who are unsure about their answers decide to not respond rather than to show their ignorance. We return to this issue below and explore the bias it might cause in our results.

With regard to generalizability to the general population, our sample includes more individuals with university education than does the population between 40 and 67 years of age (53.2% compared to 31.2%). Our sample also includes somewhat more men (54.4% compared to 50.9%). We believe that these differences are mainly due to general difficulties in recruiting people to participate in surveys, but cannot rule out that the web recruitment of the sample might have amplified the differences.

We study a total of six dependent variables – three of these variables concern knowledge of the pension system and the recent reform, while three concern preferences regarding retirement and the uptake of pension. The variables are described in detail in the notes accompanying the tables. The exact question wordings can be found in the questionnaire, which is included in Appendix C.

4 Empirical strategy

We estimate two effects of information on knowledge and preferences. First, we estimate the intention-to-treat-effects (ITT). If treatment is randomized, the ITT is simply the differences in means on the dependent variables between those in the treatment group and those in the control group. In our ITT analysis, we therefore include all the respondents in the treatment group, irrespective of whether they claim to have read the information material or not. The ITTs are interesting from a policy perspective since they estimate the causal effect of distributing the information brochure.

Next we estimate the treatment-effect-on-the-treated (TOT), which is the effect of treatment on those who actually were treated (‘the compliers’). The TOT differs from the ITT unless everyone in the treatment group was exposed to the intended treatment and no one in the control group was exposed to the treatment. We understand treatment as recent exposure to the brochure and therefore assume the latter to be true, but examine whether treatment effects are sensitive to exclusion of cohorts exposed to the brochure in 2010. Regarding whether those in the treatment group were exposed to treatment, we have information on whether they complied with the treatment allocation and actually read the information brochure. We can therefore rely on the Instrumental Variable strategy of using the treatment status as an instrument for whether the information brochure was read or not. If, and only if, treatment status is randomized, the IV estimates will be equal to the TOT. From a policy perspective, the TOTs are informative of whether readers learn from reading the brochure.

Table 1 presents descriptive statistics of the background characteristics of the two groups. The treatment and control groups are similar according to some of the background characteristics, but there are statistically significant differences on several variables, most notably on gender, income, and occupational sector (private or public). These differences cast considerable doubt on the randomization of the treatment. Presumably, the lower response rate in the treatment group has resulted in samples that are not totally random, i.e., we have a selection out of treatment.

Table 1.

Descriptive statistics

To formally assess balance between the treatment and control group, we follow Imai (Reference Imai2005) and regress treatment status on all control variables. Although randomization does not ensure balance on covariates in any finite sample, we follow convention and expect none of these variables to predict treatment. A residual deviance test indicates however that the control variables help predict assignment to treatment with p=0.0006.

To remedy the unsuccessful randomization of treatment status, we turn to matching techniques as an addition to regression models. The goal of matching is to remove the distributional differences on pre-treatment variables between the treatment and control groups (‘balancing’ the groups), so that controlling for these variables becomes unnecessary. In exact matching, the goal of balance is achieved by removing observations in the treatment (control) group for which there are no similar observations in the control (treatment) group. However, the more common matching strategy is to match treated units with control units using an estimate of the distance between the observations.

We rely on coarsened exact matching, CEM (Blackwell et al., Reference Blackwell, Iacus, King and Porro2009; Iacus et al., Reference Iacus, King and Porro2012). The CEM algorithm recodes the data into small groups and performs exact matching within stratas of the data. While exact matching removes all imbalances between the treatment and control groups, CEM allows small differences within the stratas of the data. The algorithm produces an estimate of the level of imbalance that can be compared with the pre-matching level of imbalance. If imbalance is substantially reduced, the estimates of the treatment effects are less dependent on decisions such as what controls to include and the functional form of relationships in the model, making the results less model dependent (Iacus et al., Reference Iacus, King and Porro2012). That said matching, like regression, assumes selection on observables to uncover causal relationships. This is a strong assumption since the main worry is usually about selection on unobservables. We do not know whether selection on unobservables is improved by matching, thus one should be careful when giving the estimates a causal interpretation.

5 Empirical analysis

5.1 Estimation of the ITT effects

Table 2 reports the regression results. For consistency, we estimate conventional standard errors rather than standard errors adjusted for heteroscedasticity, since the latter are not available in the matching models we present later. The conventional and the adjusted (robust) standard errors are extremely similar in those models where they both can be estimated.

Table 2.

Intention-to-treat-estimates. Linear probability models on pre-matching sample

Note: Standard errors in parentheses. ***p<0.01, **p<0.05. Treatment=1 if respondent was allocated to the group who received the brochure, 0 otherwise. Higher if retire later =1 if the respondent correctly states that her monthly pension payments will be higher if she decides to retire later in life, 0 otherwise. Lower if life exp. increases =1 if the respondent correctly states that her pension will be lower if the life expectancy of her cohort increases, 0 otherwise. Pension unaffected by unemp. =1 if the respondent correctly states that her pension is unaffected by increases in the unemployment rate of her age group, 0 otherwise. Will combine =1 if the respondent states that she will or probably will take the opportunity to work and get pension at the same time, 0 otherwise. Decided when to retire =1 if the respondent states that she has decided at what age she will retire, 0 otherwise. Planned age of retirement=The planned retirement age (if one is stated). See Appendix C for complete question wordings.

The estimates in Table 2 suggest that those who were provided with the information brochure are significantly more likely to answer the questions on the pension system correctly. The largest point estimate is for the question on whether monthly pension payments will be higher if he/she decides to retire later in life, where those in the treatment group are 10% points more likely to answer correctly (75% versus 65%). This effect is quite large as it amounts to 22% of its standard deviation. However, there are no significant differences on the pension preference questions, and the estimated coefficients are small in magnitude. The ITT estimates are similar if we include a vector of control variables (see Table A2 and Table A3 in Appendix A) or if we estimate probit models for the binary-dependent variables (Table A1, in Appendix A). Finally, one might argue that the treatment differs for those above 64 years of age since all of them received the brochure in 2010, implying that the treatment is a reminder rather than providing new information. The results are, however, identical if we remove those above 64 years of age from the sample. This is true for the analyses below as well.

We showed in the previous section that there are strong reasons to suspect that treatment is not randomized, which implies that the estimates in Table 2 are not the true ITT effects. The difference in response rate in the treatment and control group is the likely source of imbalance between the groups. One useful exercise is a bounding exercise where we examine how many of those who did not reply who had to answer the knowledge questions wrong in order to render the treatment effects insignificant. We find that about 65% of the non-repliers had to answer the retire late question wrong to make the treatment effect on that question insignificant at the 5% level. Since only 36 in the control group answered that question wrong, we find it unlikely that selection on unobservables can explain the entire treatment effect for this variable. Furthermore, 60% had to answer the life expectance question wrong, which is more conceivable given that 50% in the control group answered that question wrong. Finally, 55% had to answer the unemployment question wrong, which is probable given that 52% in the control group answered this question wrong.

Next we turn to matching to explore whether the treatment estimates are robust to a more flexible form of control for the control variables. We match on all variables listed in Table 4, all of which are binary indicators.Footnote ⁴ First, we estimate the overall imbalance of our sample (Blackwell et al., Reference Blackwell, Iacus, King and Porro2009; Iacus et al., Reference Iacus, King and Porro2012), which is the estimate we want to reduce. The overall imbalance varies between 0 (perfect overall balance) and 1 (complete separation), and can be considered as a measure of the total imbalance between the groups. Blackwell et al. (2009: 531) suggest thinking of the overall imbalance measure in the same way as R², i.e., the absolute values have less meaning than comparisons across samples.

In our case, the overall imbalance before matching is estimated to 0.66, which is reduced to approximately 0 after matching. The huge reduction in imbalance comes at the cost of efficiency, as the number of observations is reduced to 1,040. A simple placebo analysis further suggests that matching might improve our treatment effect estimates. Using the pre-matching sample, we find that those in the treatment group are about 3.5% points more likely to answer a simple math question about interest rates correctly,Footnote ⁵ a difference which is statistically significant at the 10% level. There is no reason why the brochure should make those in the treatment group more likely to answer this question correctly, thus the difference probably reflects selection bias. In the post-matching sample, however, the difference is reduced to less than 0.5% points and becomes insignificant.

The regression results after matching are presented in Table 3. The coefficients do not change much compared to those before matching, but the coefficients for two of the knowledge questions are larger. None of the differences are statistically significant. Finally, the decrease in sample size increases the standard errors, and the treatment effect on the relationship between unemployment and monthly pension is now significant only at the lenient 10% level (t=1.67). As to be expected after matching, including the vector of control variables does not change the estimated treatment effects.

Table 3.

Intention-to-treat-estimates. Linear probability models on post-matching sample

Note: Standard errors in parentheses. ***p<0.01, *p<0.1. Treatment=1 if respondent was allocated to the group who received the brochure, 0 otherwise. Higher if retire later =1 if the respondent correctly states that her monthly pension payments will be higher if she decides to retire later in life, 0 otherwise. Lower if life exp. increases =1 if the respondent correctly states that her pension will be lower if the life expectancy of her cohort increases, 0 otherwise. Pension unaffected by unemp. =1 if the respondent correctly states that her pension is unaffected by increases in the unemployment rate of her age group, 0 otherwise. Will combine =1 if the respondent states that she will or probably will take the opportunity to work and get pension at the same time, 0 otherwise. Decided when to retire =1 if the respondent states that she has decided at what age she will retire, 0 otherwise. Planned age of retirement=The planned retirement age (if one is stated). See Appendix C for complete question wordings.

5.2 Estimation of the TOT

The ITT estimate can differ substantively from the true effect of receiving treatment if there are many non-compliers in the treatment groups, i.e., respondents who are allocated to the treatment group but refuse to receive treatment. Approximately 80% of those in the treatment group claim they read the information brochure. This high rate of compliance, assuming sincere reporting, suggests that the ITT estimate and the TOT should not differ much. Since compliance is not perfect, the TOT will, however, be larger than the ITT.

We use an instrumental variable approach to estimate the TOT (Angrist and Pischke, Reference Angrist and Pischke2009, 161ff). We use the treatment indicator as the instrument for the binary variable of whether the brochure was read or not. We rely on the sample after matching. As expected, we have no weak instrument problems, as the F-value from the first-stage is 2,043 (1,179 for the ‘planned age’ variable).

The results are reported in Table 4. The TOT is, as expected, larger than the ITT, and reveals large knowledge effects of reading the information brochure. Those who read the brochure are 18% points more likely to answer the question regarding the relationship between retirement and monthly pension correctly, and 12%points more likely to answer the question about life expectancy correctly. We believe that this is strong evidence that the information brochure is successful in providing useful information that the public understands. As before, we find no treatment effects on the retirement plans.

Table 4.

Treatment effect on the treated. Instrumental Variable regressions on post-matching sample

Note: Standard errors in parentheses. ***p<0.01, *p<0.1. The instrument in the first stage is the treatment group indicator =1 if respondent was allocated to the group who received the brochure, 0 otherwise. The F stat is the Cragg–McDonald F-statistic. Read=1 if the respondent states that she has read the brochure, 0 otherwise. Higher if retire later=1 if the respondent correctly states that her monthly pension payments will be higher if she decides to retire later in life, 0 otherwise. Lower if life exp. increases =1 if the respondent correctly states that her pension will be lower if the life expectancy of her cohort increases, 0 otherwise. Pension unaffected by unemp. =1 if the respondent correctly states that her pension is unaffected by increases in the unemployment rate of her age group, 0 otherwise. Will combine =1 if the respondent states that she will or probably will take the opportunity to work and get pension at the same time, 0 otherwise. Decided when to retire =1 if the respondent states that she has decided at what age she will retire, 0 otherwise. Planned age of retirement=The planned retirement age (if one is stated). See Appendix C for complete question wordings.

5.3 Heterogenous treatment effects

In the final section, we examine whether the treatment effect is heterogeneous across education, working in the private vs. public sector, and age.Footnote ⁶ We consistently rely on the post-matching sample, and only report the ITT-estimates (The TOT-estimates can be found by multiplying the ITT by 1.25). We do not analyze treatment heterogeneity for the ‘planned age to retire’ question due to the small sample size for this variable.

There are strong reasons to expect heterogeneous treatment effects by education level. Mastrobuoni (Reference Mastrobuoni2011) argues that low-educated people have weaker incentives to gather information, since higher mortality and higher disutility from work, for instance due to health issues, instill a belief that it is optimal to retire as soon as possible. Hence, low-educated people are less likely to be willing to pay the costs of acquiring more information. Consistent with this argument, education is strongly related to knowledge about the pension system in our data (not shown). Our survey experiment is thus an information shock especially for low-educated respondents, implying that they should respond more strongly to the treatment, at least in terms of increased knowledge. That said, education is correlated with cognitive abilities, which, due to the complexity of the pension system, might reduce the treatment effect among the low educated.

We split the sample into two roughly equal groups according to level of education, with high education referring to respondents with education above high school level. We find quite large differences in the treatment coefficient for two of the knowledge questions, but surprisingly, they go in opposite directions (Table 5). We find that the treatment effect is larger for those with low education on the question about the relationship between retirement age and monthly pension, however, the difference is not significant at conventional levels (p=.15). The treatment effect on the question on life expectancy and monthly pension is however completely driven by those with a high level of education (p<.01). This difference might reflect that the issue of life expectancy is somewhat more cognitively challenging than the issue of retirement age.

Table 5.

Intention to treat-estimates by education, sector, and age in post-matching sample

Note: Standard errors in parentheses. ***p<0.01, **p<0.05, *p<0.1. Treatment=1 if respondent was allocated to the group who received the brochure, 0 otherwise. Higher if retire later=1 if the respondent correctly states that her monthly pension payments will be higher if she decides to retire later in life, 0 otherwise. Lower if life exp. increases =1 if the respondent correctly states that her pension will be lower if the life expectancy of her cohort increases, 0 otherwise. Pension unaffected by unemp. =1 if the respondent correctly states that her pension is unaffected by increases in the unemployment rate of her age group, 0 otherwise. Will combine =1 if the respondent states that she will or probably will take the opportunity to work and get pension at the same time, 0 otherwise. Decided when to retire =1 if the respondent states that she has decided at what age she will retire, 0 otherwise. Planned age of retirement=The planned retirement age (if one is stated) See Appendix C for complete question wordings.

Next we split on sector since the strongest immediate change in incentives to postpone retirement due to the reform is the ending of subsidization of early retirement for those working in the private sector with access to the early retirement scheme AFP. Early retirement is still subsidized for those in the public sector, implying a high implicit tax on work. Thus there are reasons to expect strong treatment effects on planned labor supply for those in the private sector, but not for those in the public sector. We find indications that the treatment effects on knowledge is larger among those in the public sector (Table 3, but we cannot rule out that the sector differences are sample-specific (the p-values on differences are 0.18, 0.43, and 0.18, respectively). There are no treatment effects on the preference questions.

Finally, it is plausible that treatment effects vary by age. On the one hand, the incentives to understand the pension system is stronger among those closer to the retirement age, which suggests that the treatment effect should be larger among the older respondents Mastrobuoni (Reference Mastrobuoni2011). On the other hand, the older respondents might already be better informed about the system, so that the marginal effect of additional information is smaller. In our data, older respondents are more likely to answer the ‘higher if retire later’ and ‘pension unaffected by unemployment’ questions correctly, but not the ‘lower if life expectancy increases’ question (not shown).Footnote ⁷ Nonetheless, we do not find strong heterogeneity in treatment effects by age – splitting the sample at age 55 – and neither of the age-differences reported in Table 3 are statistically significant (the smallest p-value is 0.52).