Affect is essential to understanding the subjective well-being (SWB) and individuals’ adaptation during the life cycle (Chen, Jing, Hayes, & Lee, Reference Chen, Jing, Hayes and Lee2012; Diener, Suh, Lucas, & Smith, Reference Diener, Suh, Lucas and Smith1999). The Positive and Negative Affect Schedule (PANAS; Watson et al., Reference Watson, Clark and Tellegen1988) is probably the most widely used instrument to measure the affective component of SWB. However, the alleged independence of positive affect (PA) and negative affect (NA) is still controversial (Thompson, Reference Thompson2007; Tuccitto, Giacobbi, & Leite, Reference Tuccitto, Giacobbi and Leite2010). Paradoxically, although some evidence suggests age-group differences in affective structure (e.g., Mroczek & Kolarz, Reference Mroczek and Kolarz1998; Ready et al., Reference Ready, Vaidya, Watson, Latzman, Koffel and Clark2011), the factorial structure of the PANAS has scarcely been examined in old and very old age. On the one hand, life-span theories of emotion such as sociemotional selectivity theory (SST; Carstensen, Isaacowitz, & Charles, Reference Carstensen, Isaacowitz and Charles1999), dynamic integration theory (DIT; Labouvie-Vief & Medler, Reference Labouvie-Vief and Medler2002), or the differential emotions theory (DET; Izard, Reference Izard1977), suggest that emotions become more complex with age. Emotional complexity has been operationalized by a relatively greater independence in affect indicators (Ready, Akerstedt, & Mroczek, Reference Ready, Akerstedt and Mroczek2012) and the existence of a more diffuse affective structure in late life with more item cross-loadings in older adults than in younger adults (e.g., Carstensen, Pasupathi, Mayr, & Nesselroade, Reference Carstensen, Pasupathi, Mayr and Nesselroade2000; Kercher, Reference Kercher1992; Mackinnon et al., Reference Mackinnon, Jorm, Christensen, Korten, Jacomb and Rodgers1999; Ready et al., Reference Ready, Vaidya, Watson, Latzman, Koffel and Clark2011).
Recently, the original authors of the PANAS have studied the structure of affect considering the effects of age. As they stated “it is important to establish structural convergence in affect ratings for different age groups, because structural similarity allows for meaningful comparison across age groups and for investigation of change in affect across development” (Ready et al., Reference Ready, Vaidya, Watson, Latzman, Koffel and Clark2011, p. 784). However, with the exception of Nolla, Queral, and Miró (Reference Nolla, Queral and Miró2014), the few validation studies of the PANAS with older adults have examined the short version of the scale (e.g., Hilleras, Jorm, Herlitz, & Winblad, Reference Hilleras, Jorm, Herlitz and Winblad1998; Kercher, Reference Kercher1992; Mackinnon et al., Reference Mackinnon, Jorm, Christensen, Korten, Jacomb and Rodgers1999). Furthermore, diverse methodological issues can relate to the divergence of results in the factorial studies of the PANAS (Tuccitto et al., Reference Tuccitto, Giacobbi and Leite2010). The most important issues are related to both the method used to estimate the parameters, and to the fact of taking into account (or not) the content categories from Zevon and Tellegen's (Reference Zevon and Tellegen1982) mood checklist.
Positive and Negative Affect and Their Measurement with the PANAS
On a theoretical level, Watson and Tellegen (Reference Watson and Tellegen1985) described the structure of affects as a circumplex model where PA and NA represented the two main axes (orthogonal factors). At the empirical level, the PANAS was developed by selecting terms that were relatively pure markers of either PA or NA (Watson et al., Reference Watson, Clark and Tellegen1988). For this purpose, the authors used descriptor word clusters (10 items in four categories for PA, and 10 items in five categories for NA) detailed in the factor analyses reported by Zevon and Tellegen (Reference Zevon and Tellegen1982) allowing 13 correlated error terms. Although the PA and NA scales have shown adequate levels of internal consistency, some items had different problems. For example, the term “excited” has shown high cross-loadings in the PA and NA scales (Crawford & Henry, Reference Crawford and Henry2004; Joiner, Sandín, Chorot, Lostao, & Marquina, Reference Joiner, Sandin, Chorot, Lostao and Marquina1997; Mackinnon et al., Reference Mackinnon, Jorm, Christensen, Korten, Jacomb and Rodgers1999; Rodríguez & Church Reference Rodríguez and Church2003; Thompson, Reference Thompson2007) suggesting both positive and negative connotations. The term “ashamed” has demonstrated low factor loadings for Crawford and Henry (Reference Crawford and Henry2004), Hilleras et al. (Reference Hilleras, Jorm, Herlitz and Winblad1998), Rodríguez and Church (Reference Rodríguez and Church2003), and in the Spanish versions of Joiner et al. (Reference Joiner, Sandin, Chorot, Lostao and Marquina1997) and Sandín et al. (Reference Röcke, Li and Smith1999). The term “guilty” has not been considered a general mood, but a term used to indicate culpability after an offense (Thompson, Reference Thompson2007). The detection of problematic items has entailed persistent recommendations for new validation studies (e.g., Villodas, Villodas, & Roesch, Reference Villodas, Villodas and Roesch2011).
The most controversial topic related to the PANAS continues to be the assumption of PA and NA independence, and whether that structure remains stable in different cultures and population groups (e.g., Merz et al., Reference Merz, Malcarne, Roesch, Ko, Emerson, Roma and Sadler2013). Since the exploratory factor analysis (EFA) carried out by Watson et al. (Reference Watson, Clark and Tellegen1988) its orthogonal structure has been replicated using factorial techniques in several studies (e.g., Mackinnon et al., Reference Mackinnon, Jorm, Christensen, Korten, Jacomb and Rodgers1999; Rodríguez & Church, Reference Rodríguez and Church2003; Tuccitto et al., Reference Tuccitto, Giacobbi and Leite2010), even with old and very old samples (e.g., Hilleras et al., Reference Hilleras, Jorm, Herlitz and Winblad1998; Kercher, Reference Kercher1992). For example, Mackinnon et al. (Reference Mackinnon, Jorm, Christensen, Korten, Jacomb and Rodgers1999) found that the structural characteristics of the PA and NA short-form scales were remarkably independent of differences in age and gender. Interestingly, the correlation between PA and NA did not increase as a function of age. Similarly, findings from Tuccitto et al. (Reference Tuccitto, Giacobbi and Leite2010), testing nested models in sport athletes, supported the orthogonality of the PA and NA subscales.
Validation studies in Spanish samples also gave support to orthogonality. Authors such as Joiner et al. (Reference Joiner, Sandin, Chorot, Lostao and Marquina1997) concluded that PA and NA were relatively orthogonal, after conducting EFA and CFA in a sample of women aged 45 to 65. Although they did not compare an orthogonal model to an oblique model, they found an adequate overall goodness of fit for the orthogonal model and that a one-factor model did not show a good fit to the data. Later, Sandín et al. (Reference Sandin, Chorot, Lostao, Joiner, Santed and Valiente1999) found in university students that PA and NA were quasi-independent latent variables, and a reasonably good fit for a respecified orthogonal model. Recently (Nolla et al., Reference Nolla, Queral and Miró2014) supported findings from Joiner et al. (Reference Joiner, Sandin, Chorot, Lostao and Marquina1997) and Sandín et al. (Reference Sandin, Chorot, Lostao, Joiner, Santed and Valiente1999) using EFA techniques in a convenience sample of older adults ranging from 65 to 91 years. They also found the same factorial structure across gender.
However, there is also some evidence against orthogonality. Findings from Terraciano, McCrae, and Costa (Reference Terracciano, McCrae and Costa2003) showed no differences between the oblique model and the orthogonal two factor model, but when examining the trait vs. state version, the oblique model fitted the data significantly better for the trait version than the orthogonal model. Several goodness-of-fit indices showed that the orthogonal model from Joiner et al. (Reference Joiner, Sandin, Chorot, Lostao and Marquina1997) resulted in a poor fit.
Studies testing competing confirmatory factor analytic models showed that PA and NA were significantly correlated, and the best fit for the oblique models (Crawford & Henry, Reference Crawford and Henry2004). These findings were partially supported later by Thompson (Reference Thompson2007), although the PA and NA scales showed a similar correlation, the goodness-of-fit indices indicated that the PANAS fell marginally short of a well-fitting oblique model. Poor results were attributed to item cross-loadings above .25 and low factor loadings of some items. Similarly, Merz and Roesch (Reference Merz and Roesch2011) confirmed the best fit for a model with two correlated factors and correlated error terms derived from Zevon and Tellegen (Reference Zevon and Tellegen1982). More recently, in a CFA study using nested models, Merz et al. (Reference Merz, Malcarne, Roesch, Ko, Emerson, Roma and Sadler2013) found that a three factor model, allowing correlated error terms, provided the best fit to the data. However, according to Merz et al. (Reference Merz, Malcarne, Roesch, Ko, Emerson, Roma and Sadler2013), the two-factor models were also supported, with an oblique solution (r = –.28, p ≤ .05) that fitted significantly better than the orthogonal solution.
Emotions, Aging and Culture
The relationship between age and affect is more complex in older adults rather than in those than at other life stages (Ready et al., Reference Ready, Akerstedt and Mroczek2012; Röcke, Li, & Smith, Reference Röcke, Li and Smith2009). For decades, research has shown inconsistent findings concerning whether old and very-old adults differ in their levels of affective well-being (Charles et al., Reference Charles, Luong, Almeida, Ryff, Sturn and Love2010; Pinquart, Reference Pinquart2001; Vogel, Schilling, Wahl, Beekman, & Penninx, Reference Vogel, Schilling, Wahl, Beekman and Penninx2012).
In addition to age, both the experience and expression of emotions are influenced by cultural factors (Carstensen et al., Reference Carstensen, Isaacowitz and Charles1999). One of the biggest questions about the structure of affect is whether the structure found in individualistic cultures can be replicated in collectivistic cultures (Rodríguez & Church Reference Rodríguez and Church2003). Theoretical approaches such as Adams, Anderson, and Adonu’s (Reference Adams, Anderson, Adonu, Haskek and Aron2004) about the cultural component of emotions, and studies of Diener and Suh (Reference Diener, Suh, Kanneman, Diener and Schwarz1999), and Mesquita (Reference Mesquita2001), showed intercultural differences in the experience of emotions that make it necessary to continue examining their internal structure. For example, Shimmack, Oishi, and Diener (Reference Schimmack, Oishi and Diener2002) examined whether gender moderated the cultural effect on the relationship between pleasant and unpleasant emotions. They observed gender differences in the experience of emotions: while in Western cultures the women´s correlation between PA and NA was more negative than the men´s correlation, in Asian cultures, the correlations were identical for women and men. To date, the results about the relationship between the PA and the NA in individualistic and collectivistic cultures have been inconsistent and further studies are needed.
Present Study
Surprisingly, there are no studies about the psychometric properties of PANAS in people aged 60 and over in Spanish older adults using CFA techniques, although this instrument is widely used to measure subjective well-being in this population (e.g., Márquez-González, Izal, Montorio, & Losada, Reference Márquez-González, Izal, Montorio and Losada2008). So, Spain is a collectivistic country that represents a new ground to study the validity of the PANAS. The invariance of the measure has not been explored either, although some authors have drawn attention to the lack of validation studies of well-being scales with this kind of sample (e.g., Godoy-Izquierdo, Moreno, Pérez, Serrano, & García, Reference Godoy-Izquierdo, Moreno, Pérez, Serrano and García2013; Nolla et al., Reference Nolla, Queral and Miró2014).
Thus, the aims of this study were to examine: (a) competing models of the internal structure of the PANAS proposed by Crawford and Henry (Reference Crawford and Henry2004), Tuccitto et al. (Reference Tuccitto, Giacobbi and Leite2010) and Sandín et al. (Reference Sandin, Chorot, Lostao, Joiner, Santed and Valiente1999); (b) the measurement invariance across age and gender; (c) the reliability of the PA and NA; and (d) its criterion validity and convergent/discriminant validity. In contrast to previous studies we did not examine a unidimensional model.
Considering Ready et al.’s (Reference Ready, Vaidya, Watson, Latzman, Koffel and Clark2011) findings with older adults and the research about emotions in different cultures, we expected a better fit for orthogonal models and a less clear factorial structure than in studies of individualistic cultures. Regarding the invariance of the PANAS, we were interested in testing whether participants from different age groups (young-old: 60–74 vs. old-old: 75+) and genders (males-females) employed the same conceptual framework to respond to the scale (configural invariance), whether the item loadings of the PA and NA latent factors were invariant (metric invariance), and whether the latent factors have the same relationship in all groups (covariance factor invariance). Similarly to Thompson (Reference Thompson2007), and considering the main tenets of developmental theories about emotion regulation in old age, we tested criterion validity examining the influence of age and gender on PANAS scores. We predicted that PA would be correlated positively with age and inversely with NA (Charles, Reynolds, & Gatz, Reference Charles, Reynolds and Gatz2001), and that women would score higher on NA and lower on PA than men (Crawford & Henry, Reference Crawford and Henry2004; Lim, Yu, Kim, & Kim, Reference Lim, Yu, Kim and Kim2010). Concerning the relationship with convergent/discriminant validity, we predicted that PA would be positively related to satisfaction with life, and negatively to depression and loneliness, and that NA would be positively related to depression and loneliness, and negatively to satisfaction with life (Chen et al., Reference Chen, Jing, Hayes and Lee2012; Merz & Roesch, Reference Merz and Roesch2011; Ready, Weinberger, & Jones, Reference Ready, Weinberger and Jones2007; Terracciano et al., Reference Terracciano, McCrae and Costa2003).
Method
Participants and Procedure
Participants were Spanish community-dwelling older adults living in Salamanca that were recruited through a stratified sampling with equal allocation according to age and sex based on the municipal census. The response rate was 66.77%. The results reported here refer to 585 participants (M = 74.11, SD = 7.92, range = 60–98). Women represented 53.5% of the sample. The most frequent marital status was married (56.9%). Most of the sample (86.2%) had children (M = 2.61), was living with his/her partner (56.4%), and had primary or secondary education (73.5%). Nearly 40% of the sample was earning less than 451€ per month. Data were gathered by face-to-face interviews in their private homes. Participants were not paid. The interviews lasted approximately an hour. After obtaining the informed consent, the anonymity and confidentiality of participants and of the data were assured. The study was approved by the Ethics Committee of the Universidad de Salamanca.
To perform the invariance analysis we split the sample in two groups according to their age and gender: the “young-old” (60–74 years, n = 292, 53.4% female and 46.6% male) vs. the “old-old” (75 years and over, n = 293, 53.6% female and 46.4% male), and male (n = 272, aged 60 to 98, M = 74.32, SD = 7.08) vs. female (n = 313, aged 60 to 98, M = 73.94, SD = 7.79).
Instruments
The Positive and Negative Affect Schedule (PANAS)
(PANAS: Watson et al., Reference Watson, Clark and Tellegen1988; Spanish version by Sandín et al., Reference Sandin, Chorot, Lostao, Joiner, Santed and Valiente1999). As in the original PANAS, the Spanish PANAS is a 20-item inventory with 10 items for PA and 10 for NA. We instructed participants to rate the extent to which they experienced each emotion “during the past week.” Items were rated on a 5-point Likert-type scale, from 1 (not at all or very slightly) to 5 (very much) (for PA: M = 33.02, SD = 8.24, and for NA: M = 19.50, SD = 6.29).
Satisfaction with Life Scale (SWLS)
(SWLS: Diener et al., Reference Diener, Emmons, Larsen and Griffin1985, Spanish version provided by Diener, Reference Diener2009). We used this scale to measure the cognitive component of happiness. The scale is designed to enable individuals to evaluate their lives (an overall evaluation of one’s life) according to their own subjective criteria. The scale consisted of five items (e.g., “In most ways, my life is close to my ideal”), which were rated on a 5-point Likert scale from 1 (strongly disagree) to 5 (strongly agree). The sum of the items forms the total score of life satisfaction. The reliability was satisfactory (α = .85).
De Jong Gierveld Loneliness Scale (DJGLS)
(DJGLS: De Jong Gierveld & Kamphuis, Reference De Jong Gierveld and Kamphuis1985; Spanish version by Buz & Pérez-Arechaederra, Reference Buz and Pérez-Arechaederra2014). The scale is composed of six items phrased negatively and five items phrased positively. The positive items express feelings of social embeddedness (e.g., “There are plenty of people I can lean on when I have problems”). The negative items express feelings of desolation (e.g., “I often feel rejected”). These items had three response categories (3 = Yes, 2 = More or less, 1 = No) that must be dichotomized (De Jong Gierveld & Van Tilburg, Reference De Jong Gierveld and Tilburg2011). If the response More or less or Yes is given to a negatively worded item a scale point is assigned to the emotional loneliness score. The same rule is applied if the response More or less or No is given to a positively worded item. The final score ranges from 0 = no loneliness to 11 = extreme loneliness. The scale has been used in several surveys and has proved to be a rather robust, reliable and valid instrument. The measure was found to be reliable in this sample (α = .79).
Geriatric Depression Scale Short-Form (GDS-8)
(GDS-8: Buz, Reference Buz1996). We measured the presence of depressive symptoms with the 8-item version of the Geriatric Depression Scale (Yesavage et al., Reference Yesavage, Brink, Rose, Lum, Huang, Adey and Leirer1982). The items may be answered yes or no. The total score ranges from 0 to 8 points, with higher scores indicating more depressive symptoms. The reliability for the scale was satisfactory (α = .93).
Data Analyses
Hierarchical confirmatory factor analysis
According to the recommendation of Jöreskog and Sörbom (Reference Jöreskog and Sörbom1996) to estimate the parameters in our CFA models, we analyzed the polychoric correlation matrix using the diagonally weighted least squares (DWLS) estimation, and we computed standard errors from the asymptotic covariance matrix. We conducted a series of CFA’s on five different models representing the full version of the PANAS to determine the competing model that best fit our data. These competing models (see Table 1), were tested without modifications.
Tested factor models and their description
To examine how each CFA model provided validity evidence we used several goodness-of-fit statistics. Because the data were nonnormally distributed (Mardia´s coefficient = 43.01, p ≤ .001) we used the Satorra-Bentler scaled χ2 (S-Bχ2). Additionally, we used (a) the root mean square error of approximation (RMSEA < .06 for acceptable fit, and between .08 to .10 for marginal fit); (b) the standardized root mean square residual (SRMR close to .08 for acceptable fit); (c) the comparative fit index (CFI > .90 for good fit); (d) the non-normed fit index (NNFI > .90 for good fit).
To test the orthogonality of PA and NA and the effects of correlated error terms in competing nested models we used the χ2-difference model comparison test (i.e., ∆S-Bχ2 [∆df, p]) adjusted for nonnormality under DWLS estimation. Rejection of the null hypothesis in the χ2-difference model comparison tests implies that the less constrained model provides a statistically significant decrease in χ2 and therefore fit the data better than its nested model. Complementarily, because the χ2 difference test is sensitive to sample size and departures from normality (Cheung & Rensvold, Reference Cheung and Rensvold2002) we used ∆CFI and ∆RMSEA (Chen, Reference Chen2007). Models were considered acceptable when both indices met cut-off values of ∆CFI < .01 and ∆RMSEA < .015. When the best fitting model was identified, we examined its model parameter estimates to determine the adequacy of an item as an indicator of PA or NA.
Measurement invariance (MI)
We conducted a multigroup confirmatory factor analysis (MGCFA) to test measurement invariance and structural invariance of the best fitting model across age and gender. In accordance with Vandenberg and Lance (Reference Vandenberg and Lance2000), we carried out a series of hierarchically ordered steps. Once adequate model fit was established for each group separately (e.g., 60–74 vs. 75+), we began with the establishment of a baseline model (the reference model) followed by testing for increasingly more stringent levels of constrained equivalence (the comparison model). For configural invariance, no equality constraints were imposed either on item parameters or on factor variances or latent means across the two groups. The next step was to impose equality constraints on the factor loadings across the groups to test metric invariance. Investigating metric invariance we can determine whether the same unit of measurement is being used for the items across the groups. The final step was to assess whether the covariance between the latent variables was equivalent across the groups. When factor covariance invariance is met, it suggests that all latent factors have the same relationship in all groups. The same procedure was repeated for gender.
To determinate whether model´s invariance constraints were likely to hold or not we also used the χ2-difference model comparison test (i.e., ∆S-Bχ2 [∆df, p]) (Satorra & Bentler, Reference Satorra and Bentler2001) as well as ∆CFI and ∆RMSEA. A difference in CFI of .01 or less was adopted as evidence that the imposition of additional constraints did not appreciably reduce the fit of the model, thus supporting the invariance hypothesis (Cheung & Rensvold, Reference Cheung and Rensvold2002).
Criterion validity and convergent/discriminant validity of the PANAS was examined through linear and non-linear relations with sociodemographic variables and other related constructs such as depression, satisfaction with life and loneliness.
All statistical analyses were conducted with SPSS 19, FACTOR 8.1 (Lorenzo-Seva & Ferrando, Reference Lorenzo-Seva and Ferrando2006) and LISREL 8.80 (Jöreskog & Sörborm, Reference Jöreskog and Sörbom1996).
Results
Hierarchical Confirmatory Factor Analysis
Similarly to previous authors (e.g., Terraciano et al., Reference Terracciano, McCrae and Costa2003), several NA scores, especially “guilty” and “ashamed”, were positively skewed and/or excessively kurtotic.
Table 2 presents the goodness-of-fit statistics of the tested models. Results showed that the models allowing correlations between error terms apparently had a better fit than the models not allowing them. According to the cut-off values for RMSEA, SRMR, CFI and NNFI, the original model of Watson et al. (Reference Watson, Clark and Tellegen1988) presented a marginal fit, and Sandín et al.´s (Reference Sandin, Chorot, Lostao, Joiner, Santed and Valiente1999) model, which permitted some correlations between error terms, only had an acceptable fit.
To determine the independence of the latent factors, we examined whether the orthogonal models (model 2a, model 2c and model 2e) had a better fit than those which allowed the factors to correlate (model 2b and model 2d) (see Table 4). The χ2-difference model comparison test revealed that the oblique models were slightly better than their respective nested models.
As expected, the latent factors PA and NA were quasi-independent (Φ = –.03). Later, we tested the effect of allowing error terms to correlate. Inferential statistics showed that a significant fit improvement only appeared when the two compared models included error term correlations. The statistical significance improvement is outstanding for the fit of model 2c and model 2d when compared to their nested model (model 2e). Considering the theoretical ground of the PANAS schedule, the correlation between PA and NA latent variables, and the cut-off values in RMSEA, SRMR, CFI and NNFI, we identified model 2c as the best fitting model.
Goodness of fit indices and confidence intervals of tested models
Note: p < .001 for the S-Bχ2 statistic in all cases.
CEs = correlated errors; S-Bχ2 = Satorra-Bentler scaled χ2; df = degrees of freedom; RMSEA = root mean square error of approximation; CI = confidence interval; SRMR = standardized root mean square residual; CFI = comparative fit index; NNFI = non-normed fit index.
Difference tests comparing nested models of the PANAS
Note: ∆S-Bχ2 = χ2-difference model comparison test. If the p value is significant it means that the extended model is better than the nested one. ΔCIF = comparative fit index difference test; ΔRMSEA = root mean square error of approximation difference test.
The next step was to examine factor loadings and error correlations to determine whether the model 2c could be interpretable at a theoretical level. Although we observed adequate factor loadings for PA (λinterested = .64, λalert = .87; λattentive = .67; λexcited = .74; λenthusiastic = .73; λinspired = .64; λdetermined = .77; λstrong = .84; λactive = .87), the term “proud” showed a very low loading (λ = .10). Also, six out of eight error correlations were not statistically significant. With respect to NA, the factor loadings were more homogeneous (λdistressed = .70; λupset = .60; λguilty = .51; λashamed = .33; λhostile = .59; λirritable = .61; λnervous = .71; λjittery = .55; λscared = .55; λafraid = .51), but also lower than those for PA. Nevertheless, all error correlations (except one) were significant. Taken together, results revealed that, especially for PA, the error terms reproduced partially Zevon and Tellegen´s (Reference Zevon and Tellegen1982) mood content categories. Moreover, the modification indices (MIs) did not show significant sources of misfit for the model. Nevertheless, the MIs suggested a cross-loading for “proud” upon PA as well as NA. We will discuss this finding later.
Measurement Invariance
Once adequate fit was established for each group separately (60–74 vs. 75+; males vs. females) measurement invariance and structural invariance were tested in model 2c throughout the imposition of increasingly stringent cross-group equivalence rules. The results showed that the configural model fitted the data reasonably well across age (RMSEA = .067, CFI = .956), and gender (RMSEA = .064, CFI = .961) (see Table 4). The metric invariance model showed that the factor loadings were invariant across age and gender. In both cases, the changes in fit indices were smaller than the cut-off values. Similarly, the factor covariance invariance test showed the equality of factor covariance across age and gender.
Goodness of fit indices for MI model comparisons across age and sex
Note: S-Bχ2/df = Chi-square to degrees-of-freedom ratio (values should be less than 3). RMSEA = root mean square error of approximation; CI = confidence interval; SRMR = standardized root mean square residual; CFI = comparative fit index; NNFI = non-normed fit index. ∆S-Bχ2 = χ2-difference model comparison test; ΔCIF = comparative fit index difference test; ΔRMSEA = root mean square error of approximation difference test.
Criterion Validity, and Convergent/Discriminant Validity
Next, we examined criterion validity. We conducted three hierarchical regression analyses to test the effects of age on the PA scale. In the first step we entered age, and in the second and third step, their quadratic and cubic functions, respectively. The procedure was repeated for the NA scale. Similarly to Crawford and Henry (Reference Crawford and Henry2004), our findings revealed that the relationship between age and both PA and NA was better described as a linear correlation. Thus, partially supporting our predictions, the correlation between age and PA was non-significant, r = –.02, but between age and NA was low but significant, r = .10, p = .05, probably due to the effect of the sample size. Additionally, univariate analyses showed no significant differences across gender on the PA scale, F(1, 583) = 1.95, p = .12, η2 = .003, but females scored slightly higher than males on the NA scale, F(1, 583) = 13.95, p ≤ .05, η2 = .023.
The correlations conducted to test convergent validity revealed, as predicted, that PA was positively related to life satisfaction, r = .42, and negatively to depression, r = –.46, and loneliness, r = –.31. Similarly, as predicted, NA was related positively to depression, r = .41, and loneliness, r = .24, and negatively to life satisfaction, r = –.21.
Moreover, we evaluated the independent contributions of PA to life satisfaction and of NA to depression and loneliness by means of three hierarchical regression analyses. For life satisfaction, we introduced in the first step NA scores, and in the second PA scores. The same procedure was repeated for depression and loneliness, but entering PA in the first step and NA in the second. Our findings revealed that both PA and NA contribute independently to life satisfaction, depression and loneliness (p < .001 in all cases). As expected, PA explained a significant amount of variance of satisfaction with life, ΔR 2 = .17, p < .001, and NA added a significant amount of explained variance to loneliness, ΔR 2 = .05, p < .001, but especially to depression, ΔR 2 = .16, p < .001.
Reliability of the Measures
The reliability estimates for PA and NA subscales were calculated using the alpha coefficient based in a non-linear approximation. We selected it instead of Cronbach´s alpha because of the ordinal nature of the variables, the asymmetry of the distributions, and the high variability in the intercorrelation of the items. For PA, reliability was .93 and for NA was .83.
Discussion
The objective of this work was to examine the internal structure of the Spanish version of the PANAS applied to individuals over 60 years old. Although this instrument is being widely used for research and clinical purposes, we have inconsistent evidences regarding its internal structure. The validation studies of the 20-item PANAS scale in older adults are very scarce. To fill this gap, we compared in a sample of community-dwelling older adults five different models to determine the independence of PA and NA, the presence of Zevon and Tellegen´s (Reference Zevon and Tellegen1982) mood content categories, the measurement invariance, the reliability, as well as the criteria, convergent and discriminant validity of the scale.
Our CFA results confirmed that the orthogonal model allowing 13 correlated error terms according to mood content categories from Zevon and Tellegen´s (Reference Zevon and Tellegen1982) (model 2c), best fit the data of Spanish older adults. These results were consistent with previous studies (e.g., Carstensen et al., Reference Carstensen, Pasupathi, Mayr and Nesselroade2000; Ready et al., Reference Ready, Akerstedt and Mroczek2012; Ready et al., Reference Ready, Weinberger and Jones2007; Tuccito et al., Reference Tuccitto, Giacobbi and Leite2010) that hypothesized an independent relationship between PA and NA in older people reflecting the complexity of affect in late life.
Regarding the presence of the emotion category system of Zevon and Tellegen (Reference Zevon and Tellegen1982), we did not obtain the expected results. Even though including error term correlations produced a significant improvement in the tested models, that structure was only partially present. The reason is that while almost all correlations between errors were significant in NA, almost none of them were significant in PA. This suggests that, although theoretically justified, the effects of allowing error term correlations seem to be more related to a methodological issue than to the fact that older Spaniards share the emotional category structure proposed by Zevon and Tellegen (Reference Zevon and Tellegen1982).
The CFA for PA and NA at the item level indicated that the majority of the items present adequate factor loadings, and only two items (“ashamed” and “proud”) did not show adequate loadings. Likewise previous authors (e.g., Joiner et al., Reference Joiner, Sandin, Chorot, Lostao and Marquina1997; Thompson, Reference Thompson2007; Villodas et al., Reference Villodas, Villodas and Roesch2011), the term “proud” was not explained well enough by PA, which questions if this item is a good indicator of PA in older adults (e.g., Thompson, Reference Thompson2007) and in those with a medium-low education level (e.g., Joiner et al., Reference Joiner, Sandin, Chorot, Lostao and Marquina1997). Similarly, Merz et al. (Reference Merz, Malcarne, Roesch, Ko, Emerson, Roma and Sadler2013) found low factor loadings and a minor threat for measurement invariance across age for the same item. In Spanish, the positive or negative meaning of proud depends on the communicative context and, it is possible that its social acceptance is different in collectivistic and individualistic cultures. As in younger populations (Thompson, Reference Thompson2007; Tuccito et al., Reference Tuccitto, Giacobbi and Leite2010) and in collectivistic cultures (Moriondo, Palma, Medrano, & Murillo, Reference Moriondo, Palma, Medrano and Murillo2012; Rodríguez & Church, Reference Rodríguez and Church2003) the term “ashamed” has some difficulties as a good indicator of NA in older adults (Kercher, Reference Kercher1992; Mackinnon et al., Reference Mackinnon, Jorm, Christensen, Korten, Jacomb and Rodgers1999). Similarly, Nolla et al. (Reference Nolla, Queral and Miró2014) found the lowest factor loadings for the terms “ashamed” and “guilty” in Spanish older adults.
From the life span perspective, our factor loadings can be reflecting improvements in age-related emotion regulation. For example, “shame” would be an emotion that the organism tries to maintain at a low level to avoid negative consequences for the person. In this line, Watson, Wiese, Vaidya, and Tellegen (Reference Watson, Wiese, Vaidya and Tellegen1999) pointed out that it is normal that NA levels were low because in everyday life old people are less exposed than other age groups to situations that produce NA. They would only increase significantly when some activating situation appears so, when the situation is solved they go back to the baseline level. Moreover, older people are more likely to reappraise a stressful event in a positive way than younger persons because they use antecedent-focused strategies more frequently than response-focused strategies to regulate their emotions (Charles & Carstensen, Reference Charles, Carstensen and Gross2007).
Results obtained from the test of measurement invariance confirmed that the two-independent factor structure (with correlated errors) can be used across young-old and very old people, as well as, across males and females. Configural, metric and factor covariance invariance demonstrated that for older adults of different ages, and for males and females (a) the PANAS measures the same construct; (b) the 20 items of the scale measures PA and NA in the same way, (c) and that the latent factors correlate similarly. This is an important issue because “the failure of a psychological measure to be equal across groups may indicate that the language used in the items or the values or aspirations in the items do not validly apply to different groups” (Atienza, Balaguer, & García-Merita, Reference Atienza, Balaguer and García-Merita2003, p. 1259). Future research should compare factor structure across different health conditions (objective and subjective), living conditions (living with a partner vs. living alone) and also explore the clinical usefulness of the PANAS in old age.
Our findings about the criterion validity of the PANAS seem to agree with cross-sectional and longitudinal results (e.g., Carstensen et al., Reference Carstensen, Pasupathi, Mayr and Nesselroade2000; Charles et al., Reference Charles, Luong, Almeida, Ryff, Sturn and Love2010; Kunzman, Reference Kunzmann2008; Windsor & Anstey, Reference Windsor and Anstey2010) that suggest the stability of affects in the elderly. However, contrary to the proposals of Pinquart (Reference Pinquart2001), and Vogel et al. (Reference Vogel, Schilling, Wahl, Beekman and Penninx2012) the present findings do not support a non-linear hypothesis about the relationship between age and affect.
Results related to the convergent and discriminant validity of PANAS permit us to recommend this instrument as a measure of subjective well-being. The relationships of the other component of well-being, life satisfaction, were positive for PA and negative for NA. In addition, as suggested by Crawford and Henry (Reference Crawford and Henry2004), the PANAS could be also used as a complementary measure to diagnose depression. Supporting the tripartite model (e.g., Clark & Watson, Reference Clark and Watson1991) high levels of depression correlated positively with NA and negatively with PA.
This work has some limitations and we consider that our results must be considered with some caution. Even though we found an orthogonal model with a satisfactory adjustment, and several theories related with the interpretation of these results, we consider that the Spanish version of the PANAS should be improved reviewing some items to better capture the structure of emotions in older adults. Several strategies can be employed to pursue this objective. For example, research advances in the expression of emotions show that the Spanish terms used to describe basic and universal emotions are organized consistently following a salience order (Delgado, Reference Delgado2009). Some of these emotions, such as “happiness” are not present in the PANAS instrument. Another promising line of research could be to examine the tenability of the three factor models (PA: NA; Afraid, Upset) studied by Gaudreau, Sanchez, and Blondin (Reference Gaudreau, Sanchez and Blondin2006) in young samples, or the more recent bifactor models argued by Ebesutani et al., Reference Ebesutani, Smith, Bernstein, Chorpita, Higa-McMillan and Nakamura2011).
Given the advantages of shortening measurement scales for older adults, similarly to Merz et al. (Reference Merz, Malcarne, Roesch, Ko, Emerson, Roma and Sadler2013), new research should examine the psychometric properties of the existing short forms of the PANAS or develop new versions using the most recent methodological trends. In this regard, we encourage future research to deeply study the functioning of this scale with studies from the Item Response Theory (IRT) (e.g., Pires, Filgueiras, Ribas, & Santana, Reference Pires, Filgueiras, Ribas and Santana2013) since IRT modeling provides a comprehensive and accurate evaluation of item characteristics based on “item-free” and “person-free” estimations.
The authors thank anonymous reviewers for helpful comments on an earlier version of this article.
This research was supported by the Educational and Culture Council of the Comunidad Autónoma de Castilla y León, Spain (Ref: SA015A-08).
