Participants

Measures

The ECR (Brennan et al., Reference Brennan, Clark, Shaver, Simpson and Rholes1998) is a self-completed questionnaire consisting of 36 items, each scored on a 7-point scale. The instrument yields scores on two 18-item scales, named anxiety and avoidance. People scoring high on the avoidance scale tend to avoid emotional closeness and intimacy, do not feel comfortable opening up to or depending on their partner, and are reluctant to ask their partner for comfort, advice, or help. People scoring high on the anxiety scale tend to be preoccupied with their romantic relationships, worry about being abandoned, desire to be very close to their partner, and ask the partner for more feeling and commitment. We used the validated Italian version of the instrument (Picardi et al., Reference Picardi, Vermigli, Toni, D’Amico, Bitetti and Pasquini2002), developed using multiple translators, iterative back-translation, and pilot testing in order to achieve conceptual, semantic and content equivalency of the translated instrument. In the original validation study (Brennan et al., Reference Brennan, Clark, Shaver, Simpson and Rholes1998), the internal consistency of ECR scales as measured by coefficient alpha was found to be .91 for anxiety and .94 for avoidance. In the present study, the alpha coefficients for anxiety were .90, .86, .90, and .92 for the non-clinical, psychiatric, dermatological and neurological sample, respectively. The alpha coefficients for avoidance were .92, .89, .89, and .89 for the non-clinical, psychiatric, dermatological and neurological sample, respectively.

Modeling procedures

In the present study, we used an integrative model that allows the analysis of the measurement invariance of the ECR, while controlling for method effects (Figure 1). In particular, we submitted the ECR scale to a Confirmative Factor Analysis by following the procedures recommended by Shaver and colleagues (Alonso-Arbiol, Balluerka, Shaver, & Gillath, Reference Alonso-Arbiol, Balluerka, Shaver and Gillath2008).

Figure 1.

The combined ECR model

Building on this work, we built three parcels of items for the avoidance subscale and three parcels for the anxiety subscale. A parceling approach is optimal for the current research because: (a) of the small size of some samples and the relative large number of indicators; (b) the approach allows for the modeling of measurement error in the ECR measure with a minimal amount of extra parameters, and (c) the main focus of the current study is the stability of the ECR scores, rather than the exact relationships among items. Moreover, with small sample sizes, item parceling is likely to increase the stability of the parameter estimates, to improve the variable to sample size ratio, and to reduce the effects of non-normality (see Little, Cunningham, Shahar, & Widaman, Reference Little, Cunningham, Shahar and Widaman2002). Use of item parcels or composites instead of items is not novel in the study of measurement invariance (see, for instance, Van de Vijver & Leung, 1997). Accordingly, items were randomly combined into three parcels of six items, separately for each dimension (Little et al., Reference Little, Cunningham, Shahar and Widaman2002). In doing so, we followed a suggestion by Little et al. (Reference Little, Cunningham, Shahar and Widaman2002), which advocated the use of three parcels because fewer parcels lead to local under-identification and more parcels lead to local over-identification. In general, for our objectives, a just-identified latent variable is preferable to an over-identified one (Little et al., Reference Little, Cunningham, Shahar and Widaman2002).

Thus, the final model was composed by six manifest indicators (parcels), represented by the individuals’ mean scores on six items belonging to one of the two ECR scales (see APPENDIX 1). Three parcels were for anxiety, three parcels were for avoidance. Each manifest indicator was allowed to load simultaneously on two latent variables. The first was the factor representing avoidance or anxiety, the second was a general factor representing the amount of variance introduced by the effect of the assessment procedures (i.e. the common method bias). Whereas anxiety and avoidance were allowed to correlate, the common method factor (henceforth CMB) was specified uncorrelated with each of them. Under this model, the proportions of variance attributable to anxiety, avoidance and method effect can be partitioned and quantified. More precisely, considering only a group at a time, the decomposition of the observed scale becomes:

(1) $$\sigma _{\rm{y}}^2 = [(\Sigma {\rm{\lambda }}_{{\rm{i1}}} )]^2 + [(\Sigma {\rm{\lambda }}_{{\rm{i2}}} )^2 + (\Sigma {\rm{\lambda }}_{{\rm{i3}}} )^2 + 2\phi _{23} (\Sigma {\rm{\lambda }}_{{\rm{i2}}} )(\Sigma {\rm{\lambda }}_{{\rm{i3}}} )] + [(\Sigma {\rm{\theta }}_{\rm{i}} )],$$

where ɸ₂₃ is the correlation between anxiety and avoidance factors, λ_i1 is the loading for the CMB factor, λ_i2 is the loading for the anxiety factor, λ_i3 is the loading for the avoidance factor, and θ_i the vector of measurement errors. Notice that the variance of anxiety and avoidance factors cannot be separated. In the above formulation, ${\rm{\sigma }}_{\rm{y}}^2$ represents total variance, [(Σλ_i1)]² represents the variance due to CMB factor; [(Σλ_i2)² + (Σλ_i3)² + 2ɸ₂₃(Σλ_i2)(Σλ_i3)] represents the variance due to anxiety and avoidance, and [(Σθ_i)] represents the variance due to measurement error. Accordingly, the ratio P_mbf = [(Σλ_i1)]² / ${\rm{\sigma }}_{\rm{y}}^2$ represents the proportion of variance explained by the CMB factor, whereas P_an+av = [(Σλ_i2)² + (Σλ_i3)² + 2ɸ₂₃(Σλ_i2)(Σλ_i3)] / ${\rm{\sigma }}_{\rm{y}}^2$ represents the proportion of variance explained by anxiety and avoidance together. Finally P_ER = [Σθ_i] / ${\rm{\sigma }}_{\rm{y}}^2$ represents the proportion of variance explained by measurement error.

As anxiety and avoidance factors are correlated, their respective variance proportions cannot be separated at the scale level. Accordingly, two variance components can be compared: trait variance (i.e. anxiety + avoidance) and method variance.

Steps of invariance analysis

In a first step, we examined the factor structure of the measures of the ECR model (Figure 1) separately in each group, using linear confirmatory factor analysis (CFA). In a second step, after the fit of the ECR model was established separately for each of the four groups, we used multigroup CFAs to examine measurement invariance. Following recommendations by Vandenberg & Lance (Reference Vandenberg and Lance2000), a sequence of nested models was tested. In the First (unconstrained) model (Table 1, Model 1), the factor loadings, the item intercepts, and the error variances were allowed to differ across groups (configural invariance). This level of invariance is an application to factorial invariance of the Thurstone’s principle of simple structure, stating that items comprising the measurement instruments should exhibit the same configuration of salient and insalient loadings across groups. In a third step (Model 2), the factor loadings (λ_i) for both anxiety and avoidance were constrained to be equal across groups (metric invariance). Because the factor loading represents the strength of the linear relation between each factor and its associated items (Bollen, Reference Bollen1989), the equivalence of loadings of the items on the underlying factor suggests that the observed score differences on an item can be meaningfully considered indicative of cross-group differences in the underlying construct. In a fourth step (Model 3), we constrained the loadings for the CMB factor to be also invariant. This level of invariance extends the equivalence of score differences at the level of CMB factor. In a fifth step (Model 4), we maintained the restrictions of the fourth step and imposed the additional constraint of equal first-order intercept invariance (i.e, scalar invariance: equal τ). This latter level of invariance is of special interest, because it is required for comparing latent means across groups (Vandenberg & Lance, Reference Vandenberg and Lance2000), and refers to equality of scale’s origin between groups. In other words, observed mean differences on the manifest variables imply that there are group differences on the latent means. Otherwise, mean level comparisons will be biased.

Table 1.

Tests of measurement invariance of the ECR scale across clinical and non clinical groups

Statistical analyses

All models were tested using Mplus 4.01 (Muthén & Muthén, Reference Muthén and Muthén2004). The following criteria were employed to evaluate the goodness of fit: chi-square likelihood ratio statistic, Tucker Lewis fit index (TLI), the comparative fit index (CFI), the root mean square error of approximation (RMSEA) with associated confidence intervals, and the standardized root mean square residual (SRMR). The chi-square statistic is sensitive to large sample sizes and easily produces a statistically significant result (Bollen, Reference Bollen1989). Thus, we accepted model resulting in CFI values greater than .90 (Bentler, Reference Bentler1990), RMSEA values lower than .08 (Browne & Cudeck, Reference Browne, Cudek, Bollen and Long1993), and SRMR values lower than .06 (Hu & Bentler, Reference Hu and Bentler1999).

To test differences among the base model and the more restricted models, we calculated restricted chi-square tests (Δχ²) with an alpha level of .05 (Bollen, Reference Bollen1989). If a step resulted in a significant chi-square difference test, the particular restriction was rejected and parameters with the highest modification index (MI; Muthén & Muthén, Reference Muthén and Muthén2004) were sequentially estimated. When partial measurement invariance was established, the sequence of models was continued.