2 The data

In order to understand the role of information and priming on the support for reforms of the pension system, we conduct two randomized controlled trials. One targets Portuguese voters (i.e. citizens with 18 years old or more) and the other focuses on university students, from the three major public universities in the Portuguese capital, Lisbon: Universidade Nova de Lisboa, Universidade de Lisboa and Universidade Técnica de Lisboa.Footnote ² In both experiments, we conduct a survey with six blocks of questions, where the first block relates to individual preferences over the pension system, the second assesses knowledge about the Portuguese SS, the third ascertains willingness to support different reforms that increase the sustainability of SS, the fourth looks at support for the convergence of the public servants pension system with SS and the fifth and sixth cover, respectively, behavioral and demographic questions.Footnote ³

In the first experiment, and following Boeri and Tabelinni (Reference Boeri and Tabelinni2012), half of the sample is randomly presented with an informative text with factual information about SS (the text is presented in Appendix A). This information is displayed upfront, before the beginning of the questionnaire and with no option to return to the text after starting to fill-in the survey.

In the second experiment, we randomize the order of blocks: some respondents are immediately asked about their willingness to support reforms (blocks 3 and 4) while others are first asked about preferences (block 1) and pension system features (block 2). After finishing one block, it is not possible to return to it at a later stage.

Both experiments are conducted using an online survey built in Qualtrics Software and spread via e-mail and social networks.Footnote ⁴ In the two cases, there is a monetary incentive, namely the possibility to win one out of three 25 euro (25€) gift cards from a major supermarket chain upon completion of the questionnaire.

The first experiment presented in this paper (on the effect of information) was conducted in June and July 2013. We collected 429 replies, with an average response time between 25 and 30 min. As expected with an online survey, our sample of voters is not representative of the population – the respondents are in general younger and with higher income than the underlying population. This lack of representativeness decreases the external validity but does not hamper the validly of the experiment results, i.e. the internal validity.

The second experiment we explore (on the effect of priming) was conducted in May 2013 and 450 replies were collected. The average response time is somewhat lower (between 20 and 25 min) as the questionnaire for students is slightly shorter. Universidade Nova is overrepresented in our sample, as are females. Again, as for the previous sample, conclusions from the priming experiment are still valid (internal validity).Footnote ⁵

Appendix B presents a description of the variables used in this paper, together with the descriptive statistics. In general, there are no significant differences between the treated and the control groups. However, for the first experiment (and excluding the information variables, which will be discussed in the next section), the treated sub-sample presents significantly less retired respondents. Also, the treated sub-sample presents somewhat less older and low-income respondents. In the case of the students’ sample, the only variable with statistically significant differences is the number of respondents from Universidade de Lisboa, higher in the case of the treated sub-sample.

For both experiments, it is important to note that the number of responses submitted is higher for the control group that for the treatment group, even though the treatment is randomly applied to half of the respondents who open the questionnaire. In particular, it is 15% higher in the case of the information experiment and 8% higher for the priming experiment. This can signal higher attrition rates for the treated population. Indeed, and focusing on the information experiment, people that open the questionnaire and see a page of text – the treatment – may be discouraged from replying. From the analysis of Table A.1, there is evidence that older, retired, lower-income individuals are more likely to quit the survey, as they are somewhat under-represented in the treated group as compared to the control group. However, given that only the replies from the respondents that concluded the questionnaire (and thereby pressed the button submit in the end of the survey) are received, it is not possible to more thoroughly assess possible selection issues. Nevertheless, given the low differences between the total numbers of respondents in each group, we are reasonably confident that selection effects play a very limited role, if any.

3 The role of information

As explained in the previous section, in the first experiment we display a factual text with information about SS to a random subset of the respondents (the treated group). The research questions that interest us are as follows: Does exposure to this text impact the respondent's reply? Does the respondent become more willing to support reforms to the system as compared with those not provided with the text? Are information levels affected by this factual text?

Respondents are posed different questions aimed at capturing their pensions literacy. The list of information variables used in the study is presented in Table 1. For each question, there are different response options.

Table 1.

List of information variables

Table 2 presents the mean values of the information variables in the sample, for questions whose answer is provided in the factual text. As there are six questions that fall in this category, Info text goes from 0 to 6 and represents the number of questions correctly answered. The remaining variables are the components of Info text and are equal to 1, if the answer provided is correct and 0, otherwise. The figures for those variables are thus the share of respondents that provided a correct answer. Table 3 presents similar information but for questions whose answer is not provided in the factual text (nine questions). The variable Info out text thus goes from 0 to 9.

Table 2.

Correct responses to questions for which the answer is provided in the text (average values)

Notes: ** (*) Statistically different from the average values of the control group at 95% (90%).

Table 3.

Correct responses to questions for which the answer is not provided in the text (average values)

Notes: ** (*) Statistically different from the average values of the control group at 95% (90%).

Two results emerge. First, the factual information is only affecting the replies to the questions that can actually be replied based on the text: in Table 3 there is no difference between the level of information of the treated and control groups, thus supporting the relevance of our treatment. Second, those treated see their information levels increased but not all questions are similarly affected.

As expected, and by assessing the time one spends in the informative text section, it is possible to conclude that not all respondents who have a chance to read the text actually do it (we consider that time spent is a good proxy of having read or not the text). It is thus instructive to look at the results in Table 4, which further decomposes the results by time. We distinguish three groups: the least treated (remain in the page for <10 s) and that amount to 30% of the treated respondents; the most treated (remain in the page for 90 s or more) and that constitute 21%; and those in between (49%), the somewhat treated. Table 4 shows that spending more time reading the text increases the information level of the respondent. Comparing the most treated with the control group, the treatment significantly increases information levels for all relevant variables. The exception is the question on the SS fund, which is the last piece of information in the text and thus faces a higher probability of being skipped, even for the most treated. The information level on the ratio of old to working age population, which is presented in the beginning of the text, is increased even for the least treated. The information about the pay-as-you-go (PAYG) nature of the system is the most widely known fact for all respondents.

Table 4.

Correct responses to questions for which the answer is provided in the text – by intensity of treatment (average values)

Notes: ** (*) Statistically different from the average values of the control group at 95% (90%); ‘Least treated’ means that the respondent spends <10 s in the page displaying the text, ‘somewhat treated’ means that the time spent is more than 10 s but <90 s and ‘most treated’ means more than 90 s in the page.

With an online survey, one may wonder if those that spend more time replying to information questions score better since, if this is so, it can be that people browse the internet looking for information; however, we have several reasons to believe that this is not the case (and it should be noted that not all questions could be answered with a quick internet search). In general, the respondents that spend more time in the information block perform better; however, (i) spending more time reading the questions may simply signal that the respondent is reading them carefully and thinking about the reply; (ii) it can also translate higher availability of time; actually, the ones that spend more time in the information block, also spend more time in the remaining blocks, where the gain of browsing the internet is nilFootnote ⁶ ; and (iii) finally, in the treated group, the average time spent in the information block is higher for the ones that spend more time reading the text. All these arguments make us confident that the results are not distorted by the possibility of browsing the internet. Also, if this were the case, our results would provide a lower bound on the impact of information.

The results in this section show that being provided with information has an impact on related pensions literacy, if one takes the time to actually read the text. But what is the implication for the support of reforms of the pension system? In the next section, we assess the support for different types of reforms that increase the sustainability of the SS system, namely by increasing the retirement age, decreasing benefits or increasing contributions.

3.1 Making SS less generous

In this section, we assess the support for the following qualitative five changes to the SS system, namely: (i) increase the retirement age; (ii) decrease pensions of the active; (iii) decrease pensions of all (active and retired); (iv) increase the contributions of the active; and (v) create or increase the contribution of the retired.

Table 5 presents the share of respondents in favor of the different reforms. There are no statistically significant differences between the control group and the least and somewhat treated. However, for the most treated, the chance of favoring some reforms is significantly higher: almost half of the respondents that actually read the text are willing to increase the retirement age, one out of three would accept a reduction in future pensions of the currently active and 37% would support the decrease of the pensions also affecting the ones already retired. These results indicate that access and willingness to read specific information about the features of the system has an impact in the support for reform. In the remainder of this section, we look further into those reforms that seem to be affected by the information provided in the factual text.

Table 5.

Share of respondents in favor of the different reforms, by intensity of treatment

Notes: ** (*) Statistically different from the average values of the control group at 95% (90%); ‘Least treated’ means that the respondent spends <10 s in the page displaying the text, ‘somewhat treated’ means that the time spent is more than 10 s, but <90 s and ‘most treated’ means more than 90 s in the page.

At this point it is important to assess if the most treated were ex-ante more knowledgeable about the pension system than the control group or the less treated. However, the level of information for those questions not addressed in the text (measured by info out text) is not statistically different across the four groups (not treated, least treated, somewhat treated and most treated).

How large is the impact of information? Table 6 presents the results of three probit models: column 1 for the increase of the statutory retirement age, column 2 for the reduction of future pensions and column 3 for the reduction of present (and future) pensions. The results confirm that the intensity of treatment is crucial, as the treatment per se does not have a significant impact.

Table 6.

Effect of information on the probability of supporting a reform

Notes: Average marginal effects of a probit model. With text is a dummy variable and thus the marginal effect is the impact in the probability of favoring the reform, in percentage points, of a change from 0 to 1 in the value of the variable. In the case of Info out text the marginal effect represents the impact of a small change in the information level; ** Significant at 5%; * Significant at 10%. Other control variables: Private better, Union, Left, Right, Young 35, Old 55, High income, Low income, Short life expectancy, Personal pension, Covered SS, Unemployed, Student, Retired, Married, Male; Interaction terms between intensity of treatment and age with income are not significant.

We thus present the results distinguishing according to the intensity of treatment (see Table 7). The regression analysis confirms that there is no difference between the willingness to reform of the control group and the least and somewhat treated. However, as suggested by the analysis above, for the most treated there is an impact, with an increased support for each reform by around 15 percentage points. Effective treatment increases support from 34% to 51% in the case of increasing the retirement age (and thus ensuring a majority favoring the reform), from 20% to 37% in the case of the decrease of future pensions and from 26% to 43% for the reduction of current (and future) pension payments.

Table 7.

Effect of information on the probability of supporting a reform, by intensity of treatment

Notes: Average marginal effects of a probit model. Apart from Info out text, variables are dummy variables and thus the marginal effect is the impact in the probability of favoring the reform, in percentage points, of a change from 0 to 1 in the value of the variable. In the case of Info out text the marginal effect represents the impact of a small change in the information level; ** Significant at 5%; * Significant at 10%. Other control variables: Private better, Union, Left, Right, Young 35, Old 55, High income, Low income, Short life expectancy, Personal pension, Covered SS, Unemployed, Student, Retired, Married, Male; Interaction terms between intensity of treatment and age with income are not significant.

In a nutshell, our analysis shows that having or acquiring a better understanding of the functioning and challenges of the system increases the willingness to undertake reforms that improve its sustainability. It is thus important to understand the differences in the treatment intensity. The results in Table 8 provide some preliminary insights, by comparing the most treated to the other treated respondents. Older respondents and the ones with lower income are more likely to take more time in the text section. In fact, it is natural to think that pensions are a topic of higher concern for older individuals. Also, the ones with lower income rely more heavily on the SS system and this may explain why they are keener to take the time to read information presented to them. Conversely, married respondents are less likely to read the text when given the opportunity to do so.Footnote ⁷

Table 8.

Determinants of treatment intensity

Notes: Average marginal effects of a probit model. Apart from Info out text, variables are dummy variables and thus the marginal effect is the impact in the probability of favoring the reform, in percentage points, of a change from 0 to 1 in the value of the variable. In the case of Info out text the marginal effect represents the impact of a small change in the information level; ** Significant at 5%; * Significant at 10%.

4 The impact of priming

As discussed in Section 2, the second experiment focuses on a sample of University students. We randomize the order of blocks: with a probability of 50%, the respondents are immediately asked about their willingness to support reforms (blocks 3 and 4) and only afterwards prompted to respond questions on their preferences over the pension system (block 1) and their knowledge about its main features (block 2). This is our treated group. With the same probability, they are given the questions in the order presented before (blocks 1 and 2 and only afterwards blocks 3 and 4). These respondents are the control group. In neither case is it possible to return to a block after finalizing it. Our aim is to assess if the presentation order has an impact on the support outcomes.

The respondents are presented with four reforms of the pension system (similar to reforms 1–4 in the previous experiment), namely: (i) to increase the retirement age; (ii) to decrease pensions of the active; (iii) to decrease pensions of both active and retired; and (iii) to increase the contributions of the active.

Does the order of the questions have an impact on the willingness to support certain reforms of the pension system? The results (Table 9) indicate that the ones first prompted to think about their preferences (e.g. size of the system, redistribution, private pillar, etc.) and the features of the current system (e.g. contribution rate, PAYG nature, retirement age, etc.) are less supportive of reforms, in particular reforms that increase the retirement age (reform 1) and reduce the future pension benefits of the currently active (reform 2) (Table 9). This may relate to the effects described in Wilcox and Wlezien (Reference Wilcox and Wlezien1993) and Palmer and Duch (Reference Palmer and Duch2001), whereby respondents try to ensure consistency between their support for reform and their previously expressed political opinions. This consistency effect is however not clear in the sample. For instance, those that favor a smaller system or those think that private systems work better than public ones are not more prone to accept the reform when first prompted to think about these preferences.

Table 9.

Share of respondents in favor of the different reforms, by treatment

Notes: ** (*) Statistically different from the average values of the control group at 95% (90%).

The mechanism behind our results is different. We obtain a very interesting result whereby ideology mediates priming. In particular, priming effects are different according to the political ideology of the respondent (political ideology is self-reported on a scale of seven nodes, from left to right). As shown in Table 10, priming is more relevant for the extremes of the political spectrum. Looking at the respondents in the extremes, priming is stronger for the far-right. Finally, the reforms affected by priming effects are different for those in the far-left as compared to those in the far-right. It is interesting to note though that asking directly about willingness to reform (and only afterwards about preferences and knowledge about the main features of the system) increases support both for those in the far-left and those in the far-right.

Table 10.

Share of respondents in favor of the different reforms, by treatment and ideology

Notes: ** (*) Statistically different from the average values of the control group at 95% (90%).

Table 11 presents the results of a regression analysis of the probability to support a reform. Priming effects are relevant for the support of a reduction of pensions of those active (reform 2), irrespective of ideology. However, in the case of an increase of retirement age (reform 1) or a reduction of the pension of the active and the retired (reform 3), priming is only relevant for those on the far-right. Table 11 also shows that being on the far-left of the political spectrum consistently reduces support for reform.

Table 11.

Effect of priming on the probability of supporting a reform

Notes: Average marginal effects of a probit model. All variables presented in the table are dummy variables and thus the marginal effect is the impact in the probability of favoring the reform, in percentage points, of a change from 0 to 1 in the value of the variable; ** Significant at 5%; * Significant at 10%. Other control variables: Info, Private better, High income, Low income, Worker, Unemployed, Short life expectancy, Wish retire 65 plus, Youth unemployment, Married, Male, Parent retired, Parent unemployed, Parent university, UL, UTL, Economics management, Doctoral, Master, Grade.

5 Beyond information and priming

In this section, we pool the information of both samples to gain some insights on the role of self-interest (e.g. being younger or older means that individuals are differently impacted by different reforms as, for instance an increase in the contribution rate impacts more heavily a younger individual) and ideology (e.g. those reporting to be on the right of the political spectrum are expected to be more willing to retrench the system, thus reducing public intervention). As the samples are not representative, results beyond those presented in the previous sections should be interpreted with caution and can only be seen as preliminary. With these constraints in mind, the results in Table 12 indicate that, in line with existing literature, ideology plays a role in the support for the different reforms. In particular and as already mentioned in the previous section, those on the left of the political spectrum are less prone to support the different reforms.

Table 12.

Determinants of the probability of supporting a reform

Notes: Average marginal effects of a probit model. Apart from Info common, all variables are dummy variables and thus the marginal effect is the impact in the probability of favoring the reform, in percentage points, of a change from 0 to 1 in the value of the variable. In the case of Info common the marginal effect represents the impact of a small change in the information level. The reform that would increase the contributions of the retired was only included in the voters’ sample; ** Significant at 5%; * Significant at 10%; Additional control variables: Sample and Treatment.

The role of self-interest is asymmetric across reforms, which we would expect given that different reforms have a different impact and are thus subject to different personal considerations. For instance, respondents with higher income are more willing to increase the retirement age. A possible explanation may be related to job satisfaction as, in fact, the proportion of those willing to retire after 65 is higher for higher-income respondents as compared with lower-income respondents. Also, those that have a self-reported lower life expectancy are less keen to increase the retirement age, as this would have higher costs for them. Another example relates to the reduction of current (and future) pensions, where older respondents are much less keen to favor such reform as compared to younger respondents. Conversely, older workers are more prone to support increases in the contributions of the current active vis-à-vis younger respondents.

There are other elements that also play a role. In particular, male respondents are more willing to reform the system, whereas the unemployed are more likely to reject reforms that reduce the generosity of the pension system (possibly because they are beneficiaries of SS and thus value it more).

Finally, our results confirm that, for some reforms, the level of information about the system is indeed relevant to increase support for reform.

6 Insights on the partial privatization of SS

Both surveys include a question to ascertain the willingness of the respondents to partially privatize the SS system, potentially at a cost (based on Schokkaert et al., Reference Schokkaert, Verhue, Pepermans, Pestieau, Gevers, Ginsburgh, Schokkaert and Cantillon2000). A proposal in this direction was put forward by the current Portuguese government, for further discussion (although it gained no political or social traction). The results of this section can provide some initial insights on the way these measures are perceived and the implications of the different implementation features, with the caveat that the samples used are constructed for two specific experiments and are not representative of the population.

The question on privatization may be found in Appendix C. Five privatization options are provided: option A is designed to entail no cost for the respondent, in the sense that the cut in contributions is proportional to the cut in benefits; options B to E entail growing embedded costs, as the cut in benefits is more than proportional to the cut in contributions. In this initial specification, the reduction in the contribution reverts to the respondent, via her monthly salary, and she is free to decide what to do with it.

As the interpretation of the question is not straightforward, in the analysis of this section we focus on those that gave consistent replies, namely by respecting monotonicity. It is interesting to note that, despite the complexity and length of the question, 75% and 82% of the respondents, respectively, for the voters and the students’ samples, give consistent replies. Nevertheless, the fact that the replies are not consistent for 18%–25% of the respondents hints to the relevance of communicating well the proposed reforms, in order to ensure an inclusive debate and the emergence of informed decisions. As an illustrative example, the inclusion of those that give inconsistent replies would increase the resistance to Option A by about 5 percentage points both for the voters and the students samples.

Table 13 presents the aggregate results for each sample. Two main results emerge. First, around one in every four to five respondents does not know whether they favor such reforms. This share is much higher than the one found for reforms 1–5 in the previous sections, where the percentage was around 5% (except for reform 5, related to the contribution of the retired, where the value was 12%). This may be related to the complexity of the question, but can also signal, as mentioned before, the need for more information and more discussion of this type of reform. Second, 39% and 48% of the respondents, respectively for the voters and student samples, favor a privatization of 50% of the system (option A), but this support is cut in half if the privatization would be in somewhat less favorable terms, increasing the sustainability of the SS system (option B).

Table 13.

Share of respondents in favor of the different options

Notes: The proposal entailed a reduction in contributions by half, which would be paid directly in the salary. Upon retirement, the pension would be lower, as if the worker had worked for X% of her initial salary.

When asked what the respondent would do with the extra money, close to 40% of the respondents that would accept the proposal claim they would fully save it for retirement (and an additional 50% state that they would save a large share for retirement).

Finally, the survey included a question with a proposal similar to option A but, instead of giving the money directly to the person via the monthly salary, it would automatically be put aside in an investment fund that could only be used after retirement. A total of 25%–30% of the respondents that previously agreed with the measure are not willing to accept this new alternative, where they would be forced to save until retirement via an investment fund. Conversely, 40%–50% of the respondents that previously refused Option A are now willing to accept it under these new conditions. From those that claimed that they would fully save the money for retirement under the previous formulation, only 60%–70% would accept the investment fund formulation. This may signal that, although they expect to save the money for retirement, they would like to have flexibility to use it before retirement if they so wished or needed; it may also mean that they would prefer to save it but not via an investment fund or not via compulsory savings.

The analysis of this section is illustrative of the relevance that the design of such reform has on its political sustainability.

7 Concluding remarks

Several countries are facing the challenge of ensuring political support for the needed reforms of the pension system. In this paper, we present the results of two randomized control trials that provide useful insights on the determinants of the support for reforms.

We show that, for a sample of Portuguese voters, literacy on pensions has, in line with previous literature, an impact on the agents’ willingness to support reforms that increase the sustainability of the pension system. By randomly providing information to half of our respondents, we show that access to (and willingness to acquire) information does have an impact on individual information levels and thereby on support for reforms. However, it is important to note that the intensity of treatment varies across individuals, i.e. not all respondents take the time to actually read the information provided to them: older, lower-income and non-married respondents are more prone to do so. Further research should aim at better understanding the differences between those willing and not willing to acquire information (and on ways to change the behavior of the latter) and, also, at better grasping the effectiveness of information campaigns (i.e. their power to increase information levels) for different individuals.

The second experiment highlights the relevance of priming. For a sample of Portuguese University students, we show that support for certain reforms is impacted by the order of questions. Also, these effects are more relevant for those in the extremes of the political spectrum, in particular for those on the far-right. The drivers of these priming effects should be further investigated.

Our analysis also finds a role for the other two main determinants of voters’ behavior presented in the literature, namely self-interest and ideology (beyond the priming effects discussed above). In particular, those on the left of the political spectrum are less likely to support any reform reducing the generosity of the system. The role of self-interest is, as expected, dependent on the exact reform being discussed, as different measures have different personal implications. However, one should keep in mind that the sample is not representative and any results beyond those directly related to the experiments should be interpreted with caution. A robust insight can only be achieved by using a representative sample of the population.

Finally, we provide some preliminary insights on the relevance of the design of reforms intended to partially privatize SS. The exact configuration presented to the respondents has an important impact in their decision to support or not the reform. The robustness of the preliminary results on privatization should be assessed by relying on a sample that is representative of the population.