Exploratory Factor Analysis

Exploratory Factor Analysis was conducted based on sample 1 of the present study. Before reducing the number of items, the preliminary subscale models were tested using this sample. In the first preliminary model, with all items, parallel analysis indicated that the percentage of variance explained by the fourth dimension with random data was higher than the one explained with empirical data (empirical data: 28.1%, 7.3%, 5.2% and 4.5%, random data: 5.6%, 5.4%, 5.1% and 5.0%). To elaborate the second preliminary model, four items with the highest factorial loads in a single dimension (i.e., without cross loadings), and higher communalities, were chosen. As such, the second model presented 12 items contemplating the dimensions of Self-control, Affective Approach and Empathy, but excluded the Assertiveness dimension. This subscale has been associated with important indicators of positive adjustment in adolescents, such as self-esteem and well-being (Sarkova et al., Reference Sarkova, Bacikova-Sleskova, Orosova, Geckova, Katreniakova, Klein and van Dijk2013). Thus, a final model was tested, in which four items with the higher loadings of the Assertiveness subscale were reintroduced into the analyses. The factor loadings, communalities and eigenvalues of the final exploratory factor analysis are presented in the supplementary Table, in the appendix.

Confirmatory Factor Analysis and Internal Consistency

Confirmatory factor analyses were performed using structural equation modeling. All models were estimated based on sample 2 of the present study. Initially, an attempt was made to evaluate the fit of the final model with four dimensions and 16 items, as well as the preliminary model with three dimensions and twelve items. Both models were configured with correlated first-order factors and presented the following fit indicators: Three factor model (12 items), χ²(gl) = 215.14 (51), TLI = 0.95, CFI = 0, 96, RMSEA (90% CI) = 0.05 (0.05–0.06); four factor model (16 items), χ²(gl) = 353.85 (98), CFI = 0.96, TLI = 0.95, RMSEA (90% IC) = 0.05 (0.04–0.05).

It should be noted that the fit indicators of the two models cannot be compared, since they are non-nested models with different numbers of items. Generally, however, the results indicate that both models are plausible. Additionally, the introduction of four items and one dimension was not accompanied by a drastic reduction in the quality of fit of the model. Thus, it was decided to keep the four-dimensional model for the reduction of the IHSA-Del-Prette. For model identification purposes, the variances of latent variables were set at one, so that all factor loadings could be freely estimated. Consequently, standardized and non-standardized loadings resulted in the same values. Item parameters are shown in Table 1.

Table 1.

Synthesis of the Confirmatory Factor Analysis Results (n = 1119)

Note: ^a = determination coefficients (i.e., squared loading) are presented in italics; β = standardised and non-standardized coefficient (latent variable variance was set at 1); SE = standard error of estimate; δ = threshold between categories; AVE = average variance extracted.

The loadings average (standardized regression coefficients between items and factors) was 0.63 (SD = 0.09). However, ten factor loadings presented values lower than 0.70, reducing the reliability of the latent scores. The correlations between the factors were moderate, indicating that a significant part of score variance is shared among the factors.

Regarding the endorsement of the categories, the thresholds, presented in Table 1, point to relatively easy items. For most items, participants, with a latent score only a standard deviation above the average, tend to endorse the higher Likert category. In Table 1, the values for average variance extracted (AVE) are also shown. It was found that the variances shared between the dimensions (determination coefficients) are smaller than the AVE. These results evidence the discriminant validity of the four factorial scores.

AVE was also used to evaluate internal consistency. Considering it is a reduced instrument that has few items, the AVE was expected to be greater than 0.49. With regards to the empathy and assertiveness factors, the higher factorial loads supported AVE values near the cut-off point. With regards to the self-control and affective approach factors, the AVE values are minimally acceptable, even though lower than expected, seeing that the factorial load averages of these factors were 0.58 and 0.60, respectively.

Still with regards to internal consistency, the composite reliability (CC) values were: 0.66 for self-control scores; 0.69 for affective approach; 0.78 for empathy; and 0.75 for assertiveness. These values, close to or higher than 0.70, also show evidence of internal consistency and scores reliability, when considering the reduced number of items, seeing that CC tends to result in single values due the few items.

Invariance Analysis

Based on the evidence of the four-dimensional structure of the reduced version of the inventory, it was sought to evaluate the invariance of the parameters among the different databases (consisting of samples from different years), age-group (12 to 14 years and 15 to 17 years) and gender, by means of multi-group analyses. For this, the total database, consisting of the two samples of the present study (N = 2291), was used. The results for the multi-group analyses are shown in Table 2.

Table 2.

Invariance Analysis - Multi-Group Confirmatory Factor Analysis (N = 2291)

Note: α = thresholds of items 10 to 15 freely estimated between genders male and female; b = differences between Partial Scalar and Metric models.

* (p < .05).

With regards to the groups divided by databases and age groups, the parameters of the items presented invariance. The fit differences, evaluated by the chi-squared test, were statically significant between the Metric model and the Scalar model, however the differences in CFI were small and lower than 0.01. Additionally, the differences of the McDonald’s index (ΔNCI) were lower than 0.02, which also denotes the invariance of the parameters. Though, Gamma-hat differences were slightly higher than 0.001. Thereby, the fit indices yielded diverging conclusions: CFI and NCI indicated invariance, Gamma-hat presented borderline results, and the chi-squared test rejected the model in which the thresholds were defined as invariant between the groups. It should be noted, however, that the chi-squared test is sensitive due to sample size. Additionally, when comparing the scalar and configural model adjustments, the critical difference ratio is less than 2 (for the database, Δχ² /Δgl = (907.9 – 828.5) / (252 – 196) = 1.42; for the age-group, Δχ² /Δgl = (885.5 – 784.0) / (252 – 196) = 1.81). In summary, there is evidence of the invariance of the parameters for the time (database) and for the age-group.

Regarding the gender variable, the multi-group analysis did not indicate any invariance of the thresholds, seeing that the scalar model presented a lower fit to the data when compared to the metric model. To expand this analysis, it was verified that only items 10 and 15 showed differences above 0.30 between the thresholds of the male and female groups (item 10, female: δ₁ = –0.01, δ₂ = 0.43, δ₃ = 0.97, δ₄ = 1.41; item 10, male: δ₁ = –0.38, δ₂ = 0.04, δ₃ = 0.52, δ₄ = 1.06; item 15, female: δ₁ = –1.34, δ₂ = –1.02, δ₃ = –0.58, δ₄ = –0.04; item 15, male: δ₁ = –1.42, δ₂ = –1.00, δ₃ = –0.40, δ₄ = 0.23). Considering that the thresholds are associated with the probability of endorsement, the results showed that girls require a latent score greater than the boys to endorse the high categories of item 10, and a lower latent score to endorse the higher categories of item 15. It should be noted that the differences are greater for item 10 (Δδ > 0.3) and smaller for item 15.