1. Introduction
Individuals in the United States are increasingly responsible for their own financial security after retirement. Yet, evidence shows that they have difficulty understanding complex aspects of retirement planning and that financial literacy rates are low worldwide (Klapper et al., Reference Klapper, Lusardi and Van Oudheusden2015). The result is that individuals may claim Social Security earlier, or utilize annuities less than is optimal, leading to poor financial security in later life (Lusardi and Mitchell, Reference Lusardi and Mitchell2007, Reference Lusardi and Mitchell2011; Benartzi et al., Reference Benartzi, Previtero and Thaler2011; Poterba et al., Reference Poterba, Venti and Wise2011).
One solution is to provide financial education or more information about retirement planning. In one study, individuals who felt they had enough information about Social Security claiming were also more satisfied in retirement (Rabinovich and Samek, Reference Rabinovich and Samek2018). In other studies, visual tools and narratives helped individuals improve their financial literacy in basic concepts related to financial planning (Heinberg et al., Reference Heinberg, Hung, Kapteyn, Lusardi, Samek and Yoong2014; Lusardi et al., Reference Lusardi, Samek, Kapteyn, Glinert, Hung and Heinberg2017). Heinberg et al. (Reference Heinberg, Hung, Kapteyn, Lusardi, Samek and Yoong2014) found evidence that video and written narratives were equally effective at improving financial literacy, but that videos were more effective at improving self-efficacy surrounding decisions about retirement. However, the most effective content and mode of communication are still open questions.
Two recent studies investigated understanding of annuities and Social Security by using experiments in an online panel representative of the U.S. population. Brown et al. (Reference Brown, Kapteyn, Luttmer and Mitchell2017) asked respondents to provide a lump sum amount they would be willing to pay for a permanent increase in Social Security monthly benefits, or to provide a lump sum amount they would have to be paid to accept a permanent decrease in monthly payments. Respondents provided divergent and inconsistent valuations. The implication is that many consumers do not understand annuities and are not able to value them. In a follow-up study, Brown et al. (Reference Brown, Kapteyn, Luttmer, Mitchell and Samek2019) found that inducing respondents to think jointly about the annuitization decision as well as how quickly or slowly to spend down assets in retirement led to improved respondents' valuation of annuities. In this paper, we build on Brown et al. (Reference Brown, Kapteyn, Luttmer and Mitchell2017, Reference Brown, Kapteyn, Luttmer, Mitchell and Samek2019) using an online experiment to investigate the impact of a similar consequence message on decision-making and financial literacy. Different from Brown et al. (Reference Brown, Kapteyn, Luttmer and Mitchell2017, Reference Brown, Kapteyn, Luttmer, Mitchell and Samek2019), this paper also evaluates the effect of modes of communicating the consequence message, including using video or a written narrative.
We propose that consequence messaging is a promising educational tool that can improve decision-making under uncertainty. The premise of consequence messaging is that although expected utility theory assumes that people make decisions by evaluating all possible consequences and their probability of occurrence, in complex situations that involve uncertainty, decisions are actually made without fully processing this information. A benefit of consequence messaging is that it describes the outcomes of multiple decisions under different states of the world. Hence, if individuals are asked to consider the consequences of an action, this should improve their understanding. In this sense, this paper is also related to Samek and Sydnor (Reference Samek and Sydnor2017), who use consequence graphs to help people understand the outcomes associated with different health insurance plan choices.
In this study, our first contribution is to evaluate how consequence messaging affects decision-making in the context of annuities and Social Security claiming. We chose these two concepts because both annuities and Social Security protect against longevity uncertainty and therefore could be affected by consequence messaging. That is, individuals may purchase an annuity to guard against uncertainty of outliving their savings. Social Security similarly guards individuals against outliving their savings by providing a constant stream of benefits, but claiming Social Security later in life can result in larger monthly benefits.
Our second contribution is to assess different modes of communicating consequences by comparing video and written vignettes. We chose to compare videos and written modes of communication because these were also used in Heinberg et al. (Reference Heinberg, Hung, Kapteyn, Lusardi, Samek and Yoong2014) and because these are common ways of communicating through the internet. A large literature in educational and applied psychology compares information processing with print and audio and visual mediums (Alexander, Reference Alexander2013; Furnham, Reference Furnham2019; List and Ballenger, Reference List and Ballenger2019). Several studies find participants remember information in print better than audio–visual formats (Furnham, Reference Furnham2019; List and Ballenger, Reference List and Ballenger2019) whereas other studies find only marginal differences in performance across mediums (Alexander, Reference Alexander2013). Our study contributes to this literature by directly comparing print and audio–visual mediums for teaching retirement finance concepts in a large sample of adults.
In our vignettes, a 62-year old man is talking to his financial advisor about his plans for budgeting his retirement. The financial advisor encourages the man to consider the consequences of different decisions. The financial advisor explains that outcomes depend partly on his decisions – i.e., how much money to spend down, and partly on uncertainty – i.e., the uncertainty surrounding how long the man can expect to live. The vignettes do not constitute a pure consequence message since the financial advisor also describes the basic features of the decision, for example by explaining the link between claiming age and level of Social Security benefits in the Social Security vignette.
To evaluate our vignettes, we conducted an experiment in the Understanding America Study (UAS). The UAS is a nationally representative probability-based internet panel (N = 6,000 at the time of the study) housed at the University of Southern California. We recruited nearly 2,000 participants aged 30–70 to participate in the study and randomized them between-subjects in a 2 × 3 experimental design to either the Social Security or annuities condition, and to either receive no vignette, a written vignette, or a video vignette. The written and video vignettes contained the same content but were presented either through video or as text on a webpage. In the valuing annuities vignette, the man is making a decision about whether to purchase an annuity. In the Social Security claiming vignette, the man is making a decision about when to claim his Social Security benefits.
We evaluate the impact of our vignettes on decision-making and understanding by using two main outcome variables. The first outcome is advice that the respondent would give a hypothetical person who is facing the decision of whether to annuitize or when to claim Social Security. Within-subjects, we experimentally vary whether the hypothetical person is in relatively good health or relatively poor health and evaluate this ‘spread’ in advice by treatment. We consider this variable as most representative of decision-making, since we could not actually observe respondents' own decisions in these contexts. Furthermore, asking respondents to give advice to a hypothetical person limits concerns about respondents' own wealth and health and allows us to manipulate health. The second outcome is the performance of respondents in a short quiz measuring financial literacy related to annuities or Social Security.
We find that, relative to the control group, respondents randomized to the vignettes advise significantly larger spreads of annuitization amounts and Social Security claim ages between the hypothetical person in relatively poor health and the hypothetical person in relatively good health. We take this as evidence that the vignettes affect decision-making and provide suggestive data to indicate that this change leads to improved decisions. We also find that the vignettes significantly improve accuracy of responses to the financial literacy quiz. We take this as evidence that consequence messages improve understanding of annuities and Social Security, at least in the short term. We do not find conclusive evidence that one mode of communication is better than another; however, most respondents indicate that they prefer to receive the information in written form.
In what follows, Section 2 describes our experimental design. Section 3 summarizes our results. Section 4 provides a discussion and concludes.
2. Experimental design
2.1 Vignette development
We created two vignettes about the same 62-year old man and his financial advisor. Each vignette (in video format) was about 3 min long. The first vignette focused on annuities, and the second vignette focused on Social Security claiming age decisions. The written scripts for the vignettes and links to the video version, as well as screenshots of the video, are provided in Appendix B. In both vignettes, the 62-year old man is meeting with his financial advisor to discuss his plans for budgeting his retirement.
The goal of both vignettes was to provide information about the consequences of living longer or shorter, stress the uncertainty in one's lifespan, and explain how this impacts the money that one can spend during retirement. In the annuities vignette, the financial advisor explained that an annuity acts as insurance against uncertain life expectancy: ‘Annuities are like insurance against outliving your money. You pay a premium up front, but then you're guaranteed a monthly payment until you die’. However, the financial advisor does not actually advise purchasing an annuity. In the Social Security vignette, the financial advisor explains how monthly Social Security benefits change as a function of claiming age and clarifies that one does not need to claim in the same year as one retires from work: ‘Your retirement benefits depend on the age when you begin claiming. It's a tradeoff – you can decide to claim earlier. In that case, you would have lower monthly benefits, but you'd also get to enjoy these benefits for a longer period’. However, as before, the financial advisor does not actually advise delaying claiming.
2.2 Participant recruitment
We conducted our experiment in the UAS, an online panel that is representative of the U.S. population.Footnote 1 An advantage of using this panel is that we are able to understand the impact of our vignettes on the population they are meant to affect: i.e., older adults who are facing or will face the decision to annuitize or claim Social Security. Another major advantage of using the panel is that the UAS contains rich data on socio-economic status (SES) and cognitive abilities of respondents, which can be linked to the data we collect.
We recruited a random sample of 2,150 Americans aged 30–70 from the UAS pool to participate in the study, and 1,808 respondents ultimately completed the study (84% response rate).Footnote 2 Table 1 provides summary statistics of the sample. The average age of our respondents was 52 (S.D. = 10.32). About 43% were male and 57% were female. About 87% were white, 10% were black, and 7% were Hispanic. Respondents came from a range of backgrounds. About 24% of the sample had an income of less than $30,000 per year, and 26% had an income greater than $100,000. About 25% of the sample had a high school education or less, whereas 36% had an education equal to a bachelor's degree or higher. Our data on SES and demographics comes from prior UAS surveys. As such, we are missing data on race for six respondents, and missing age for one respondent. We include missing dummies in the regressions for these respondents.
Summary statistics
Notes: This table shows summary statistics of demographics (age, gender, and race), SES (household income and highest level of education attained), order in which the scenarios were presented, numeracy, and missing data. The median age in our sample is 54, the 10th percentile is 36 and the 90th percentile is 65. The numeracy score is taken from an 8-item numeracy scale designed by Weller et al. (Reference Weller, Dieckmann, Tusler, Mertz, Burns and Peters2013), and then compiled into a single measure using an Item Response Theory (IRT) model.
2.3 Experiment procedures
The study was conducted in two waves – a smaller group was recruited in July–November 2018 and follow-up data collection occurred in April–July 2019. Our experimental design is presented in Table 2. Our 2 × 3 experimental design includes experimental variation in (1) whether respondents were asked about annuities or Social Security claiming and (2) whether respondents received no vignette (control group), a written vignette or a video vignette. Randomization was done at the respondent level, with the aim of randomizing an equal number of respondents to each of the six treatment cells. This resulted in about 300 observations in each treatment cell.
Experiment design
Notes: This table shows the number of respondents in the analysis sample randomized to each treatment.
In the vignette treatments, following the vignette we also asked respondents to indicate whether they were able to fully view the vignette. Approximately 95% of respondents in the video vignette treatments and 99% of respondents in the written vignette treatments indicated that they could view the vignette fully, suggesting that most people were exposed to the intervention as expected. Despite the fact that a small minority could not view the vignettes, we include everyone randomized to each treatment in our analysis (i.e., we perform an intent-to-treat analysis).
At the end of the survey, all respondents received a short questionnaire that assessed the impact of the vignettes on decision-making and financial literacy. The questions also asked respondents to rate their concerns and expectations about retirement planning and indicate their preferences for receiving information. The questions are available in Appendix C. First, to assess decision-making, respondents were given two scenarios about the man from the video (in random order) and asked to give advice to the man about how much annuity to purchase or when to claim Social Security.Footnote 3 The ‘long-life’ scenario described the man as being in relatively good health and expecting to live a longer life: ‘based on his family history and his relative good health, Bill expects to live at least until he is 85’. The ‘short-life’ scenario described the man as being in relatively poor health and expecting to live a shorter life: ‘based on his family history and his relatively poor health, Bill expects to live until he is around 70’. Relative to respondents in the control treatment, we expected respondents in the vignette treatments to give advice that was more responsive to the differences in the man's circumstances – i.e., to have a larger ‘spread’ between the advice in the ‘short-life’ and ‘long-life’ scenarios.
Second, to assess financial literacy related to annuities and Social Security claiming, we asked True/False questions about each concept. Respondents assigned to the annuity condition received four questions about annuities and respondents assigned to the Social Security condition received four questions about Social Security (in random order).Footnote 4 The True/False statements dealt with basic features of annuities or Social Security benefits, such as, ‘An annuity is a financial product that pays a lump sum when you die’ and, ‘You have to start claiming Social Security as soon as you stop working completely’. We expected that if the consequence message were effective at improving understanding, then respondents randomized to the vignettes would get more of these questions correct than respondents randomized to the control group.
Third, we asked respondents how much importance they place on several concerns that people may have about retirement (in random order). The concerns were related to annuities or Social Security, depending on the treatment to which the respondents were assigned. We developed these concerns with the aim of including some concerns that were related to considering consequences and some concerns that were not. Respondents were asked to value each concern on a five-point scale from ‘Not at all important’ to ‘Very Important’. For annuities, the consequence-related concerns included (1) ‘The risk of not getting to spend most of your money in your lifetime’, (2) ‘The risk of running out of money in your lifetime’, and (3) ‘Uncertainty about how long you will live’. For social security, the items were (1) ‘The risk of claiming Social Security too late and not getting to enjoy the full benefits in your lifetime’, (2) ‘The risk of claiming Social Security too early and getting a lower monthly payment during your lifetime’, and (3) ‘Uncertainty about how long you will live’. For both annuities and Social Security, the non-consequence-related concerns were (1) ‘Whether you have enough money saved up for retirement’ and (2) ‘Leaving money for your children or other dependents’ (see Appendix C for a full script).
Our hypothesis was that if the consequence message helps people consider the consequences of different outcomes, then respondents randomized to the consequence treatments would place a higher importance on the consequence-related concerns versus respondents randomized to the control group. The remaining questions asked about preferences for mode of receiving communication about annuities or Social Security, and expectations about one's own claim or annuitization decisions.
Respondents received $5 just for completing the survey and earned additional money by providing correct answers to the quiz.Footnote 5 In line with what is typically done in the UAS, respondents' earnings were deposited on their re-usable debit cards the first week of the following month.
3. Results
Our analysis includes all 1,808 respondents who completed the survey. Table 3 shows that we are balanced on all observable characteristics by treatment, suggesting that our randomization worked as intended.Footnote 6 Because respondents randomized to a video vignette were more likely to have trouble viewing their treatment as compared to respondents randomized to a written vignette, we are unbalanced on ability to view the treatment. This biases downward our chance of observing an effect of the video vignette, as those unable to see the video effectively did not receive their treatment. Given this imbalance, we include regressions in Appendix Table A.1 that exclude respondents who were unable to view the vignettes. The results are qualitatively unchanged.
Balance table
Notes: This table shows balance across treatments for demographics (age, gender, and race), SES (household income and highest level of education attained), order in which the scenarios were presented, numeracy, and missing demographic data. The final column displays the p-value from an F-test comparing all treatments. Of 1,808 respondents, 35 said they could not view vignette (2%), 6 are missing race data, and 1 is missing age. We include missing dummies for age and race.
*p < 0.10 **p < 0.05 ***p < 0.01.
3.1 Impact on decision-making
To determine whether consequence messaging might affect decision-making, we investigated the advice respondents gave in the long- and short-life scenarios. To evaluate decision-making on the extensive margin, we create a variable that takes the value of 1 if the advised amount to annuitize or the advised Social Security claim age is higher in the long-life scenario, and 0 otherwise. To evaluate decision-making on the intensive margin, we create a spread variable, which takes the difference between advice given in the long- and short-life scenarios. We expect the spread to be positive, meaning that respondents should recommend annuitizing more money and claiming Social Security later in the long-life scenario relative to the short-life scenario. We further expect the spread to be larger in the vignette treatments, suggesting that the vignettes increase the responsiveness of respondents to information that changes longevity beliefs. This would amount to a change in the elasticity of a respondent's advised annuity purchase amount or claim age with respect to life expectancy.
Under the annuities condition, 622 of 882 (71%) respondents gave a directionally correct response, and under the Social Security condition 778 of 926 (84%) of respondents gave a directionally correct response. Figure 1, panel (A) summarizes the proportion of directionally correct responses by treatment. Although the number of directionally correct responses is larger for the treatment groups, the differences are not statistically significant (p-values from chi-squared tests comparing vignettes to control are between 0.06 and 0.19).
Long- and short-life scenarios by treatment. Panel (A) Directional accuracy. Panel (B) Spread between Scenarios. Notes: This figure shows mean and standard error bars for the proportion of directionally accurate respondents (Panel A) and the raw difference in scenario recommendations (Panel B) by treatment and condition (either annuity or Social Security).
Given the high percentage of directionally correct recommendations that already exist in the control treatments, we suspect that we may face a ceiling in terms of finding treatment effects on this variable. Hence, Figure 1, panel (B) summarizes the spread variable. Under the annuities condition, the spread is measured as a percent of the total possible allotment of $250,000. The average spread under the annuities condition is 15 percentage points (S.D. = 23.9) and the average spread under the Social Security condition is 45 months (S.D. = 35.2). We observe statistically significant differences in the spread by treatment for most comparisons in the direction we would expect; i.e., the spread is larger in the vignette treatments relative to the control group. Under the annuities condition, the spread is 7 percentage points larger in the written vignette treatment relative to the control group (p = 0.002 from a two-sided t-test) and 2 percentage points larger in the video vignette treatment relative to the control group, but this latter result is not statistically significant (p = 0.233). The difference in spread when comparing the written and video vignettes is statistically significant under the annuities condition (p = 0.04 in a post-estimation Wald test). Under the Social Security condition, the spread is 11 months larger in the written vignette treatment relative to the control group (p < 0.001), and 12 months larger in the video vignette (p < 0.001). There is no statistically significant difference in the spread when comparing the written and video vignette (p = 0.715) under the Social Security condition.
Figure 2 presents kernel density plots of the distribution of the spread variable. We find that the distributions of spread are also statistically significantly different in most comparisons. Under the annuities condition, a Kolmogorov–Smirnov test comparing the distributions is statistically significant at the 1% level when comparing the written vignette to control (p = 0.003), but not when comparing the video vignette to control (p = 0.065). Under the Social Security condition, a Kolmogorov–Smirnov test comparing treatments to control is statistically significant in both the written vignette and the video vignette (p = 0.003 and p < 0.001 respectively). There are no statistically significant differences in the distribution of the spread when comparing the written and video vignette in either the annuity (p = 0.302) or Social Security (p = 0.785) treatments.
Distribution of spread between long- and short-life scenarios. Notes: This figure shows a density plot of the raw spread between the short-life and long-life scenarios for Annuities and Social Security, by treatment.
Table 4 provides regressions that confirm the results of our t-tests and show that our results are robust to the demographic, SES, numeracy, and scenario order controls that we include. Column 1 uses the binary 1/0 variable of directionally correct recommendations as an outcome variable, whereas columns 2 and 3 use the spread as the outcome variable. In column 3, we additionally control for the advice given in the short-life scenario in order to compare scenario differences with similar baseline advised annuitization amounts or claim ages. The treatment effects when comparing each vignette treatment to the control group remain large and statistically significant for all treatments except for the Annuities-Video treatment. Post-estimation tests comparing the written and video vignettes do not yield statistically significant results (p-values between 0.06 and 0.94). We find no effects of scenario order on any outcome. In Appendix Table A.1, we run the same analysis but excluding respondents who said they had problems viewing the vignette; our results are qualitatively unchanged. Appendix Tables A.2 and A.3 display coefficients for the full list of demographic and SES controls are found in Table 4.
Average treatment effects
Notes: This table shows treatment coefficients from regressions of outcome variables on the written and video vignettes, as well as controls for age, gender, race, household income, education level, numeracy, recruitment wave, and the order in which the scenarios were presented. Specification (1) uses whether or not the respondent was directionally accurate as the outcome. Specification (2) uses the raw difference in recommended investment/claim age between the long-life and short-life scenarios as the outcome. Specification (3) uses the same measure, but controls for the baseline amount in the unhealthy scenario. Specifications (4) and (5) use the percent correct on True/False questions and the difference in Likert scale valuation of consequence and non-consequence-related concerns respectively.
*p < 0.10 **p < 0.05 ***p < 0.01.
The advised spread treatment effects that we observe in column 2 are economically meaningful. For example, for a 62 year-old man, delaying Social Security claiming by 11 months results in 6.1% higher benefits each month.Footnote 7 Similarly, an investment of 5.5% of $250,000 in an annuity amounts to $13,750, which at a payout rate of 6% per year (an average market rate for a 62-year old male)Footnote 8 amounts to an additional $70 per month.
3.2 Impact on financial literacy surrounding retirement planning
We next investigate the impact of our vignettes on understanding of annuities and Social Security. The average percentage of correct True/False responses under the annuities condition was 87.4% (S.D. = 19.5) and the average percentage of correct True/False responses in the Social Security scenario was 89.8% (S.D. = 19.5). Figure 3 shows the percent correct by treatment. Relative to the control group, the vignette treatments showed a higher percent correct by about 10 percentage points for the annuities condition and 14 percentage points for the Social Security condition. We find that under the annuities condition, both written and video vignettes showed statistically significantly higher percentages of correct answers versus control (p < 0.001 for either vignette in t-tests comparisons with control). Under the Social Security condition, the written and video vignettes also showed statistically significantly higher percentages of correct answers versus control (p < 0.001 for both vignettes). The vignettes were not statistically different from each other under either condition. Table 4 presents these results as a regression in specification 4, controlling for demographics, SES, scenario order, and numeracy.
Percentage correct True/False questions by treatment. Notes: This figure shows mean and standard error bars for the percent correct on four True/False questions by treatment and condition (either annuity or Social Security).
3.3 Impact on concerns and expectations
We next evaluate whether the vignette treatments increased the importance that respondents placed on consequence-related concerns about retirement and on the respondents' reported own plans about annuitization and claiming. Although these are the variables we might ultimately wish to affect, we should note that it is generally difficult to move these variables. For example, Perez-Arce et al. (Reference Perez-Arce, Yoong and Rabinovich2019) found that an intervention with information about spousal benefits associated with Social Security claiming age affected the advice that respondents gave in hypothetical situations but did not affect respondents' own expected claiming ages. In addition, with regard to the Social Security claiming age, we did not have an a priori expectation of which direction it should move by treatment, since this would depend on self-assessed longevity. Finally, with respect to annuitization, an individuals' expected wealth at retirement and access to private pensions are unobserved variables that affect the reports.
We restrict our analysis to the 91% of individuals who gave a rating for each of the five concerns (this omits 157 respondents). The average importance placed on the three consequence-related concerns was 3.52 (S.D. = 0.89) on a 5-point Likert scale, whereas the average importance placed on the two concerns unrelated to consequences was 3.76 (S.D. = 0.93) on the same scale. The difference is small and does not seem to differ significantly by treatment.
Figure 4 displays the difference in importance placed on consequence-related concerns over non-consequence-related concerns, by treatment. When looking at annuities and Social Security separately, we find positive but not statistically significant treatment effects on consequence-related concerns over non-consequence-related concerns in the video vignette (t-test comparisons to control yield p = 0.053, p = 0.054 respectively) and the written vignette (p = 0.16, p = 0.73 respectively). When we include controls the picture is slightly improved. Column 5 in Table 4 shows significant positive effects (p < 0.05) of the video vignette on the valuation of consequence-related concerns over non-consequence-related concerns. This suggests that our video vignette was more successful in moving respondent beliefs.
Consequence-related difference. Notes: This figure shows mean and standard error bars for the Likert scale valuation of consequence-related concerns (the mean of three concerns) minus non-consequence-related concerns (the mean of two concerns). This difference is split by treatment and condition (either annuity or Social Security). The bars are negative because respondents on average rated non-consequence-related concerns higher than consequence-related concerns.
To explore the effect of our vignettes on respondent concerns, we show treatment effects on each rated concern separately in Appendix Tables A.7 and A.8. In the annuity treatments, we find that both the vignettes seemed to have increased the concern about not getting to spend all of one's money. In the Social Security treatments, the video vignette seemed to have increased concerns about claiming Social Security too late.
The treatments had a mixed impact on the respondents' expectations about their claiming age or level of annuitization. Some of our respondents may have already claimed Social Security or made decisions about annuitizing. Hence, we next restrict our analysis to respondents who are not yet old enough to claim Social Security (age less than 62). Among this group, on average respondents under the annuities condition reported a likelihood of annuitizing in the future of 2.8 (S.D. = 0.95) out of 5. The video vignette increased the likelihood over control by 0.22 points or 8.3% (t-test comparison to control yields p = 0.02), whereas the written vignette had no statistically significant effect (p = 0.08). Under the Social Security condition, respondents on average answered that they would begin collecting Social Security benefits at age 65.6 (S.D. = 2.75). Neither the written nor the video vignette had an effect on expected claiming age (p = 0.42 and p = 0.56 respectively). Appendix Tables A.9 and A.10 show these results in a regression with demographic controls in column 1. We find that the treatment effect of the video vignette on the likelihood of annuitizing in the future is robust to the inclusion of controls. Columns 2–4 explore the interaction of treatment with age, race, and gender. We find no evidence of heterogeneity of treatment effects on likelihood of annuitizing or claim age.
3.4 Further analysis
3.4.1 Welfare
Although we observed large impacts on the spread in advice by treatment, we next attempt to evaluate whether this effect reflects a welfare improvement, given the limited information we provided to the respondents about the annuity instrument and Social Security claim benefit. This analysis is exploratory, since we did not identify this as an issue a priori and since respondents were given limited information, making complex calculations challenging. Hence, we may think of this exercise as merely suggestive.
To study welfare effects, we transform respondents' advice in both scenarios into the present value of a stream of monthly payments. We take advantage of the fact that the only difference between the two scenarios is life expectancy, which increases by 15 years in the long-life scenario relative to the short-life scenario. For simplicity, we ignore the fact that there is some uncertainty in longevity in each scenario. Under the annuities condition, we calculate a monthly payout assuming a fixed rate annuity with an annual payout rate of 6%.
Under the Social Security condition, we follow the claim age tables on the Social Security website to convert the respondents' recommended claim ages into a percentage of their full monthly retirement benefit, starting at 70% if they claim at age 62, and increasing on average 0.4 percentage points each month delayed after that.Footnote 9 We allow our percent of monthly benefit value to keep increasing at this increment until 70, in order to match the possible responses in our simplified scenario. We assume the maximum 2019 full retirement benefit of $2,861 to convert this percentage value into dollar terms.
Given a fixed life expectancy, the present value of a monthly payout under either condition is as follows, where n represents the number of months of expected life left, and r is the prevailing discount rate, which we assume to be 3%:
Table 5 shows vignette treatment effects on the present value of the respondent's long-life recommendation, controlling for the short-life recommendation as well as demographics and SES. The written vignette treatments increases present value by $2,244 under the annuities condition (p = 0.003) and $2,071 under the Social Security condition (p = 0.019). The video vignette treatments show positive but not statistically significant effects. Furthermore, Wald post-estimation tests comparing written to video do not show significant differences (p = 0.093 under the annuities condition and p = 0.343 under the Social Security condition). This preliminary analysis suggests that both treatments not only increased the spread of the recommended values, but also improved welfare for the hypothetical person.
Welfare treatment effects
Notes: This table shows treatment coefficients from regressions of the present value of the long-life recommendation on the written and video vignettes, as well as the baseline present value of the short-life recommendation. All regressions control for age, gender, race, household income, education level, numeracy, and the order in which the scenarios were presented.
*p < 0.10 **p < 0.05 ***p < 0.01.
3.4.2 Interaction effects
Our final question is whether we observe heterogeneous treatment effects by age, race, and cognitive ability. We consider age because we expect that vignettes may be more effective for people who are younger and have therefore had less exposure to or thought less about annuitizing and claiming decisions. We consider race because we are interested in exploring whether the video vignette, which featured white actors, was more salient for white respondents relative to other races or ethnicities. This is related to the sociological concept of homophily, the tendency for people to seek out or trust those similar to themselves. Homophily has been shown to affect information processing, comprehension and belief in health (Boulware et al., Reference Boulware, Cooper, Ratner, LaVeist and Powe2003; Hill et al., Reference Hill, Jones and Woodworth2020; Alsan et al., Reference Alsan, Garrick and Graziani2019) and political contexts (McPherson et al., Reference McPherson, Smith-Lovin and Cook2001; Halberstam and Knight, Reference Halberstam and Knight2016). In our experiment, we might expect that respondents who perceive themselves as demographically similar to either our financial advisor or our respondent respond stronger to treatment. Finally, we consider cognitive ability because we are interested in understanding whether our treatments help narrow the gap in financial literacy between higher and lower cognition respondents.
Tables 6 and 7 explore the possibility of heterogeneity in vignette treatment effects on both our binary directionally correct variable and our spread variable. Columns 1 and 2 in both tables display the interaction of the vignettes with respondent age. Older adults are more likely to have had exposure to retirement planning concepts (and some may have already made their own decisions about annuitization or claiming Social Security) and may therefore respond less to the treatment vignettes. However, when age is entered linearly, we find that treatment effects do not vary significantly with respect to age, with the exception of a small effect on the written vignette under the annuities condition (older participants are less directionally accurate in the written vignette). In Appendix Figure A.2 we explore the age interaction further by pooling the annuity and social security conditions and running treatment effects by age quintile to consider whether there are non-linear effects. We find that although respondents in the youngest quintile (age 30–41) show larger treatment effects on directional accuracy and directional spread for both written and video vignettes, the differences in treatment effect across quintiles are not statistically significant. As a last exploration of age, in Appendix Tables A.4 and A.5 we include an interaction of treatment with an indicator for whether the respondent is 60+ years old. We find no significant effect of this treatment dummy interaction.
Interaction treatment effects: annuities
Notes: This table shows treatment-interaction coefficients from regressions of outcome variables under the Social Security condition. Treatments are interacted with respondent age (continuous), if the respondent was black (binary) and respondent's numeracy score (continuous). In each case, the first column uses whether or not the respondent was directionally accurate as the outcome. The second column uses the raw difference in recommended claim age between the long-life and short-life scenarios as the outcome. All regressions control for the written and video vignettes, as well as age, gender, race, household income, education level, numeracy, recruitment wave, and the order in which the scenarios were presented.
*p < 0.10 **p < 0.05 ***p < 0.01.
Interaction treatment effects: social security
Notes: This table shows treatment-interaction coefficients from regressions of outcome variables under the annuities condition. Treatments are interacted with respondent age (continuous), if the respondent was black (binary) and respondent's numeracy score (continuous). In each case, the first column uses whether or not the respondent was directionally accurate as the outcome. The second column uses the raw difference in recommended investment between the long-life and short-life scenarios as the outcome. All regressions control for the written and video vignettes, as well as age, gender, race, household income, education level, numeracy, recruitment wave, and the order in which the scenarios were presented.
*p < 0.10 **p < 0.05 ***p < 0.01.
Columns 3 and 4 in Tables 6 and 7 use the interaction of treatments with black and Hispanic respondents. We might expect the video vignette to be less salient for black or Hispanic respondents, as the video featured a white financial advisor and advisee, and there is some evidence in the medical field which suggests that advice is taken more seriously from professionals of the same race as the patient (Boulware et al., Reference Boulware, Cooper, Ratner, LaVeist and Powe2003; Hill et al., Reference Hill, Jones and Woodworth2020). We do not find evidence of this. Instead, we find some evidence that the treatment was more effective for black respondents. For example, we find that in the Social Security scenario the interaction of black with treatment is positive and significant (p < 0.05) for the directional accuracy outcome. However, the interactions for black or Hispanic in any other measures are not statistically significant.
Columns 5 and 6 display the interaction of treatment with numeracy.Footnote 10 We find that in the annuities scenario, respondents' numeracy skills are significantly positively associated with the impact of the treatment. However, the effect sizes are small – a 1 point increase in numeracy score is associated with a 1% improvement in probability of directional accuracy. This is the opposite of what we might have speculated, i.e., that simple vignettes may be more effective for low-numeracy respondents than high-numeracy respondents. Similar results are observed when we use a financial literacy scoreFootnote 11 or educational attainment in Appendix Tables A.4 and A.5. Respondents who scored high on financial literacy have a statistically significantly higher (p < 0.01) treatment effect in the written vignette of the annuities condition. However, no other association with financial literacy is statistically significant. Educational attainment does not appear to consistently influence treatment effects.
Finally, we explore the influence of homophily on treatment effects for more nuanced demographic categories intended to capture the characteristics of the financial advisor and retiree presented in the vignettes. Appendix Figure A.1 shows treatment effects of the written vignette and video vignette respectively by the demographic groups white male over age 54 (the median age in our sample), white male below age 54, white female over age 54, white female below age 54, and the rest of the sample (non-white, all ages). To preserve statistical power, we pool the annuities and Social Security scenarios. We find no consistent evidence that demographic similarity to either the financial advisor or the retiree improves treatment effects relative to other groups, suggesting that homophily is not a big factor in the effectiveness of the vignettes. For example, for the directional accuracy outcome, we see that white males below age 54 are more affected by the treatment, whereas white females see marginal treatment effects in the written vignette but not in the video vignette. In the comprehension outcome (percent correct), we see that non-white respondents showed the highest treatment effects relative to other groups.
4. Discussion and conclusion
The decision to annuitize and the decision for when to claim Social Security are important in the retirement planning process since they substantially affect wellbeing and economic security in retirement. Research finds that people tend to under-utilize annuities and claim Social Security earlier than is optimal. In this study, we conducted an experiment with a large representative sample of Americans to investigate the impact of short written and video educational tools on decision-making and financial literacy surrounding these complex concepts. Our survey included nearly 2,000 Americans aged 30–70 years old. We randomized respondents in a 2 × 3 design to either receive the annuity or Social Security scenario, and to either receive no vignette, a written vignette, or a video vignette. The vignettes provided information about the consequences of making different decisions about annuitization and Social Security claim age, respectively.
We found that our vignettes affected decision-making and financial literacy when measured immediately after the intervention. Our measure of decision-making was the advice that respondents gave to a hypothetical person about how much to annuitize and when to claim Social Security. We believe this measure is better than using own decisions, since we were able to experimentally manipulate the perceived health of the hypothetical person in each scenario, and since the responses could be de-coupled from other aspects of the respondents' own circumstances, such as amount of money available for annuitization. In the vignette treatments, respondents' advice was more responsive to the health circumstances of the hypothetical person than in the control group. Furthermore, in the vignette treatments, respondents answered more True/False questions correctly about the concepts than in the control group.
We found that only the video vignettes had an impact on self-reported concerns about retirement planning and on expected annuitization decisions.Footnote 12 This suggests that our video vignettes may have been more salient to respondents and therefore more effective at improving self-efficacy as suggested in some prior study with visual tools (Heinberg et al., Reference Heinberg, Hung, Kapteyn, Lusardi, Samek and Yoong2014; Brown et al., Reference Brown, Kapteyn, Luttmer and Mitchell2017; Lusardi et al., Reference Lusardi, Samek, Kapteyn, Glinert, Hung and Heinberg2017). However, the video vignette also failed to produce a treatment effect on directional spread in the annuities scenario, making it difficult to determine which vignette performed better overall.
We also collected data on respondents' preferences for receiving communications about annuities and Social Security. Respondents were given a multiple-choice question that included the following modes of communication: watch a video online, read an article online, or receive information in the mail. The most commonly-selected preference for receiving communications was receiving information in the mail (34.13% of respondents), followed by reading an article online (28.54% of respondents) and watching a video online (25.5% of respondents). We found that preferences varied by age and education. Participants over age 60 and participants with no college education were significantly more likely to prefer receiving information in the mail (see Appendix Table A.6).
Since both written and video modes of communication proved equally effective on most dimensions, the choice of which to use in practice may depend both on preferences and on cost. Disseminating videos online has a high fixed cost (i.e., to produce the video) but a low marginal cost (sending out the link). This low marginal cost means that disseminating videos online may be easy to quickly scale to many viewers. On the contrary, providing information through the mail may have lower fixed cost (no need to produce a video) but higher marginal cost (the cost of printing and postage).
Our study leaves several questions open for future research. From a theoretical perspective, research should try to better understand what it is about consequence messaging that makes it effective. This research should include considering why consequence messaging works for conveying factual information, but is not always effective at increasing concerns about consequences. It would be interesting to understand whether people place more weight on negative or positive consequences. Future research could also address other types of consequence messaging, for example, addressing risk perception in a broader range of settings. Finally, an interesting follow-up would be to investigate whether such interventions have long-lasting effects.
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/S1474747221000111.
Acknowledgements
The bulk of this research was conducted while Anya Samek and Andre Gray were at the University of Southern California. This research was partly supported by a grant from the U.S. Social Security Administration (SSA) as part of the Retirement Research Consortium (RRC). This research was also partly supported by a pilot project as part of a Roybal grant awarded to the University of Southern California, entitled ‘Roybal Center for Health Decision Making and Financial Independence in Old Age’ (5P30AG024962-12). The project described in this paper relies partly on data from surveys administered by the Understanding America Study (UAS) which is maintained by the Center for Economic and Social Research (CESR) at the University of Southern California. The opinions and conclusions expressed herein are solely those of the authors and do not represent the opinions or policy of any institution with which the authors are affiliated nor of the SSA, any agency of the Federal Government, the University of Michigan Retirement Research Center (MRRC), USC, CESR or the UAS.
