Introduction
Paranoia, the belief that others will intentionally cause one harm (Freeman & Garety, Reference Freeman and Garety2000), is common in the general population with approximately 28% of individuals reporting the elevated levels of paranoid thinking in everyday life (Freeman et al., Reference Freeman, Loe, Kingdon, Startup, Molodynski, Rosebrock and Bird2019). Paranoia can be conceptualised as existing on a continuum, ranging from mild concerns of suspicion and distrust of others, to less commonly reported delusions of persecution, which are more typical in clinical groups (Freeman et al., Reference Freeman, Garety, Bebbington, Smith, Rollinson, Fowler and Dunn2005). Deciphering the intentions of others and making judgements as to whether others can be trusted is a complex human process, which is influenced by individual and environmental factors, and their interaction. Stress-vulnerability models of paranoid thinking emphasise the key role of high stress environments (e.g. victimisation, minority status and social maladjustment), affective states (e.g. fear, anger, anxiety and depression) and cognitive mechanisms (e.g. self-representations, social cognition and mentalisation) in understanding the development of paranoid thinking (e.g. Freeman, Bentall, & Garety, Reference Freeman, Bentall and Garety2008; Garety, Kuipers, Fowler, Freeman, & Bebbington, Reference Garety, Kuipers, Fowler, Freeman and Bebbington2001; Preti & Cella, Reference Preti and Cella2010; van Os, Kenis, & Rutten, Reference van Os, Kenis and Rutten2010). According to Preti and Cella's (Reference Preti and Cella2010) human heuristic account, paranoia helps an individual manage uncertainty during stressful situations. Stress, triggered by environmental factors and affective states, is thought to enhance the probability that neutral stimuli are viewed as threatening, which functions to avoid possible harm whilst not missing possible social gain. Under these conditions, interpretations and decision making over-relies on paranoid-based interpretations of experience.
In January 2019, the corona virus disease 2019 (COVID-19) pandemic introduced a rapidly intensifying threat into the lives of people across the world, which has had a substantial effect on the physical and psychological wellbeing of many people (e.g. Chandola, Kumari, Booker, & Benzeval, Reference Chandola, Kumari, Booker and Benzeval2020; Ran et al., Reference Ran, Gao, Lin, Zhang, Chan, Deng and Liu2020). The pandemic has created an environment that contains many of the characteristics that we know give rise to paranoia in the general population such as stress, worry, social detachment and isolation, loss of work, anxiety and depression (e.g. Lamster, Lincoln, Nittel, Rief, & Mehl, Reference Lamster, Lincoln, Nittel, Rief and Mehl2016; Olfson et al., Reference Olfson, Lewis-Fernandez, Weissman, Feder, Gameroff, Pilowsky and Fuentes2002; Saarinen et al., Reference Saarinen, Grano, Hintsanen, Lehtimaki, Cloninger and Keltikangas-Jarvinen2020; Startup, Freeman, & Garety, Reference Startup, Freeman and Garety2007). Furthermore, and with reference to paranoia, it has introduced a heightened state of fear and anxiety as to whether others’ actions and intentions can be trusted. Studies examining paranoia during the pandemic have shown: that exposure to information about the pandemic is associated with paranoid thinking (Jaspal, Lopes, & Lopes, Reference Jaspal, Lopes and Lopes2020), that paranoid thinking is higher than pre-established norms in Chinese students (Ruichen, Reference Ruichen2020) and that young people, in particular, are likely to be experiencing paranoia in response to the pandemic (Lopes, Bortolon, & Jaspal, Reference Lopes, Bortolon and Jaspal2020). At the more severe end of the spectrum, Hu, Su, Qiao, Zhu, and Zhou (Reference Hu, Su, Qiao, Zhu and Zhou2020) reported a 25% increase in incident cases of psychosis compared to the previous year, which authors attributed to psychosocial stress and physical distancing.
Existing research on paranoia during the pandemic has utilised measures of paranoia that were developed prior to and thus independently of the pandemic. As such, these studies have assessed instances of general paranoia during the pandemic (i.e. people have been talking about me behind my back; others want to hurt me), rather than pandemic-specific paranoia; that is, paranoid cognitions that focus specifically on the threat posed by others to oneself because of the pandemic (i.e. others spreading rumours that I have COVID-19; others wanting to infect me with COVID-19). We propose that the specific threat of being infected is likely to be reflected in the content of paranoid concerns and that this is likely to be related to but nevertheless distinct from general paranoia. This is supported by a general population survey of catastrophic cognitions (Rosebrook et al., Reference Rosebrook, Cernis, Lambe, Waite, Rek, Petit and Freeman2021) of which some reflected paranoid concerns (i.e. others deliberately trying to give me the virus) that were moderately associated with general paranoia. A measure of pandemic-specific paranoia would have many possible benefits. Firstly, if pandemic-specific paranoia exists, a measure specifically tailored to assess this would have the necessary precision to understand and predict behaviour during the pandemic. Secondly, such a measure would enable an examination of whether, how, and for whom, the context of a pandemic has given rise to pandemic-specific paranoia and how this relates to and differs from a general tendency towards paranoia. Thirdly, it could help us to understand how pandemic paranoia relates to indices of distress and pro-health behaviours and the ways in which it may be functional and/or problematic in the context of the pandemic. Fourth, it could be used to evaluate the impact of support during the pandemic, and finally, it may help to anticipate individuals who may find increased social contacts more difficult, as restrictions ease.
To this end, the current paper describes the development and psychometric evaluation of the Pandemic Paranoia Scale (PPS), designed to measure pandemic-specific paranoia in the general population. Existing research suggests that paranoia is hierarchically structured in the general population, whereby common experiences of interpersonal worry, vulnerability, suspicion and mistrust, which are less explicitly persecutory, form the building blocks for delusional belief formation (Bebbington et al., Reference Bebbington, McBride, Steel, Kuipers, Radovanovic, Burgha and Dunn2013; Freeman et al., Reference Freeman, Garety, Bebbington, Smith, Rollinson, Fowler and Dunn2005). As the severity of threat of harm increases, thoughts become rarer and increasingly characterised by fears of intentional persecution and conspiracies. Informed by the paranoia hierarchy model and the approach to measuring general population paranoia taken by existing measures [e.g. The Revised Green et al., Paranoid Thoughts Scale (R-GPTS); Freeman et al., Reference Freeman, Loe, Kingdon, Startup, Molodynski, Rosebrock and Bird2019] the PPS was designed to assess pandemic paranoia from across the full spectrum of experience, ranging from milder suspiciousness and mistrust (e.g. ‘Other people cannot be trusted to keep our community safe from COVID-19’) to more severe ideas of intentional persecution and conspiracy (e.g. ‘I couldn't stop thinking about people wanting to infect me with COVID-19’, ‘The government is using the COVID-19 pandemic to control us’). The current paper aimed to (1) determine the factor structure and item loadings for the PPS in a first representative sample from the UK, (2) use confirmatory factor analysis (CFA) to examine the factor structure and item loadings in a pooled sample from the four remaining sites (USA, Australia, Germany and Hong Kong) including three language versions (English, German and Chinese), (3) examine measurement variance across cultures/languages, (4) examine convergence of factors on a higher order factor and (5) examine psychometric properties (validity and reliability).
Methods
Design
A cross-sectional design was employed. The full sample (i.e. UK, USA, Germany, Hong Kong and Australia) completed the PPS at baseline, alongside construct validity measures (detailed below). Ninety-eight participants from the UK sample recompleted measures 3 months later, enabling examination of test–retest reliability. The construct validity was tested by examining its convergence with trait paranoia (medium-large positive correlation predicted), COVID-anxiety (medium positive correlation predicted), conspiracy beliefs (medium positive correlation predicted) and perceived risk of infection (medium positive correlation predicted).
Participants
Participants were recruited via Qualtrics using stratified quota sampling to ensure that each sample was representative of the respective general population based on sex, age and educational attainment. Data were collected between February and March 2021. Sample size was determined using Everitt's (Reference Everitt1975) minimum ratio recommendation of 10:1 (participants:scale items) per site. A total of 2510 participants met quota and quality assurance conditions (see section ‘Procedure’). The distribution across sites was: UK (n = 512), USA (n = 535), Germany (n = 516), Hong Kong (n = 445) and Australia (n = 502).
Measures
Pandemic Paranoia Scale (PPS). An expert international panel was convened to develop the initial set of items. The initial pool of items was generated based on the existing measures of general population paranoia (e.g. R-GPTS; Freeman et al., Reference Freeman, Loe, Kingdon, Startup, Molodynski, Rosebrock and Bird2019) and aimed to capture the full spectrum of paranoia, from milder concerns about suspicion and mistrust in the context of COVID-19 to more severe fears of intentional persecution and conspiracies, whilst incorporating key dimensions (e.g. preoccupation, conviction, distress; Statham, Emerson, & Rowse, Reference Statham, Emerson and Rowse2019). An initial pool of 28 items was developed [score range 0 (not at all) to 4 (totally)], which could be clustered into the following themes: the belief that others want to infect me with COVID-19 (e.g. I was sure someone wanted to infect me with COVID-19); belief that others are intentionally spreading the virus (e.g. other people cannot be trusted to keep our community safe from COVID-19); the belief that others are talking about you/watching you in the context of COVID (e.g. people are watching me more closely due to COVID-19); distrust in the government/conspiracy beliefs (e.g. politicians are using COVID-19 to control us). Items were translated into German and Chinese for these samples. Bilingual graduate/masters level students translated the English items into German/Chinese. Items were back-translated and checked by authors. Items requiring adaptation were discussed and resolved by consensus.
Descriptive and socio-demographic variables. Participants provided information on a range of sociodemographic variables. Those reported in this study included: age, gender, household income, employment status (in the last year), country of birth and current diagnosis of a mental health disorder (yes/no).
The remaining measures were used for construct validation.
The Revised Green et al., Paranoid Thoughts Scale (Freeman et al., Reference Freeman, Loe, Kingdon, Startup, Molodynski, Rosebrock and Bird2019) is an 18-item measure comprised of two subscales: ideas of reference (8 items) and paranoia and ideas of persecution (10 items). Items are rated on a 5-point scale of 0 – not at all to 4 – totally and exhibit reliability across the paranoia continuum. In the current sample alpha was 0.959.
COVID-19 anxiety (Shevlin et al., Reference Shevlin, McBride, Murphy, Miller, Hartman, Levita and Bentall2020) was assessed using a single-item visual analogue scale. Participants were asked: ‘How anxious are you about the coronavirus COVID-19 pandemic?’ rated from 0 to 100 with 100 indicating higher levels anxiety.
Perceived risk of infection (Shevlin et al., Reference Shevlin, McBride, Murphy, Miller, Hartman, Levita and Bentall2020). Perceived risk was assessed using two items, assessing perceived probability of contracting the virus (slider of 0–100%, in increments of 10, and across three times scales – in the next month, the next 3 months and next 6 months) and perceived consequences if infected (slider of 0–100, not too bad–very bad), in increments of 10, and across the same three time scales.
Conspiracy Mentality Questionnaire (Bruder, Haffke, Neave, Nouripanah, & Imhoff, Reference Bruder, Haffke, Neave, Nouripanah and Imhoff2013) is a five item, unidimensional measure of the general tendency to believe in conspiracy theories (i.e. ‘I think that …’). Items are rated on a Likert scale from 0% – certainly not to 100% – certain. In the current sample alpha was 0.905.
Validated German and Chinese language versions of all scales were used where available. Any remaining scales and items were translated from English by bilingual graduate/master's level students and back-translated and checked by authors.
Procedure
Ethical approval was obtained from each of the five host sites. Potential participants were contacted by Qualtrics to take part. Consenting participants completed the questionnaires online via Qualtrics and were reimbursed for their time. Three months later, those opting into the follow-up received a link to the second set of measures. To prevent missing data, participants were required to respond to all questions on each page before progressing through the survey. To enhance the accuracy of the data, participants had to correctly respond to all of five attention checks that were distributed through the survey. Completion time was also monitored and those taking less than half of the median completion time were excluded. Participants with a geographical location that did not correspond with the stated location, and/or who did not consent to their data being used and/or dropped out without completing all measures were excluded at source by Qualtrics. Participants not fulfilling quota conditions were also excluded. Based on these criteria and conditions, n = 3555 participants were excluded at source by Qualtrics.
Statistical analysis
Statistical analysis was carried out using R 3.6.1 and SPSS 26.0. First, preliminary statistics were computed to assess skew, kurtosis and to report on sample characteristics.
Second, to assess the factor structure of the PPS, the UK sample was used to conduct a non-graphical analysis of the Scree plot. The number of factors for subsequent analyses was based on eigenvalues (>mean eigenvalue), parallel analysis and optimal coordinates (Raîche, Walls, Magis, Ripoel, & Blais, Reference Raîche, Walls, Magis, Ripoel and Blais2013). Only those items loading over 0.600 on a factor were retained for further analyses. Next, exploratory factor analysis (EFA) was conducted in the UK sample, using principal axis factoring to assess data-driven clustering of questionnaire items. The aim, here, was to observe the number and nature of latent factors without imposing theory driven constraints. CFA was conducted using the combined sample of the remaining four sites (USA, Germany, Hong Kong and Australia) to examine whether the factor model determined using EFA in the UK sample showed sufficient fit in this independent set of data. Indicators for model fit were: the comparative fit index (CFI) and the Tucker–Lewis index (TLI; with CFI/TLI >0.95 indicating good fit and CFI/TLI >0.90 indicating a sufficient fit), the root mean squared error of approximation (RMSEA, with RMSEA <0.06 indicating a good fit and RMSEA <0.08 indicating a sufficient fit) and the standardised root mean square residual (SRMR, with SRMR <0.08 indicating a sufficient fit). To account for the often non-normal distribution of paranoid ideation, CFA was calculated with maximum likelihood estimation with robust standard errors and a Satorra–Bentler scaled test statistic.
Third, measurement invariance was assessed across all five samples by calculating and comparing four CFA. In a first CFA, sample was entered as a group variable to test whether the proposed factor structure fitted equally to all five samples (configural invariance). In the second CFA, factor loadings were fixed to be equal across samples (weak invariance). In the third CFA, factor loadings and intercepts were fixed (strong or scalar invariance, necessary to compare latent trait values between samples). In the fourth and final steps, loadings, intercepts and residuals were fixed (strict invariance, necessary to directly compare sum-scores from different samples). Invariance was evaluated based on the established thresholds (Putnick & Bornstein, Reference Putnick and Bornstein2016) for CFI (ΔCFI >−0.01 for invariance), RMSEA (ΔCFI <0.015 for invariance) and SRMR (ΔCFI <0.015 for invariance), as well as the Akaike information criterion (AIC) and the Bayesian information criterion (BIC).
Fourth, internal consistency was assessed using Cronbach's alpha and composite reliability (ω) (Netemeyer, Bearden, & Sharma, Reference Netemeyer, Bearden and Sharma2003). Coefficients greater than 0.70 were considered acceptable.
Fifth, convergent validity was assessed (using data from all sites) by examining the covariation of the PPS scale with other related, but distinct, constructs. Good convergence was expected to occur between the PPS and (a) general tendency to experience paranoia (R-GPTS, moderate-strong correlation anticipated), (b) COVID-19 anxiety (moderate correlation anticipated), (c) perceived risk of infection (moderate correlation expected) and (d) conspiracy mentality (moderate correlation expected). Criterion validity was examined by testing whether those participants self-reporting a mental health diagnosis had elevated levels of PPS than those reporting no diagnosis.
Finally, test–retest reliability was examined in a subsample (n = 98, UK sample) using intraclass correlations (ICCs), specifically: two-way mixed effects, single measurement absolute agreement ICCs (Koo & Li, Reference Koo and Li2016), correlating baseline scores and those obtained 3-months later.
Results
Preliminary statistics
Two thousand five hundred and ten participants met quota and provided valid data. Sample characteristics are reported in Table 1. Across sites, all items were positively skewed (M skew = 1.94; range: 0.38–3.02) and most items showed excess kurtosis (M kurtosis = 3.51, range: −1.14 to 8.48). Characteristics of the test–retest UK subgroup were: M age = 47.54, 55% female, highest education (1% primary, 15% GCSE, 32% A-level, 36% Undergraduate Degree, 13% Masters, 3% Ph.D.), average household income (13% <£18 500, 39% £18 500–36 999, 21% £37 000–55 999, 12% £56 000–74 999, 6% £75 000–92 999, 4% £93 000–111 999 and 4% >£112 000), employment (45% full time, 18% part time, 17% retired, 6% unemployed, 9% home keeper and 4% disabled) and 13% reporting a current mental health diagnosis.
Sociodemographic details (age, gender, education, income, employment and mental health diagnosis) across samples
Factor analysis
Prior to EFA, the item-correlations were checked for multicollinearity (r > 0.80) due to excessive similarity in item content. Out of the 378 item correlations, nine correlations (2.3%) were above the 0.80 threshold (see online Supplementary Table S1 for a detailed list). Of these nine item pairs, five showed no substantial item overlap. Item content for the remaining four items was deemed sufficiently different to warrant including all items in the analysis (e.g. items 11 and 12 shared the same scenario – being targeted to be infected. However, item 11 ‘I couldn't stop thinking about people wanting to infect me with COVID-19’ focused on preoccupation, whereas item 12 ‘I was distressed by being targeted by people who wanted me to catch COVID-19’ focused on distress associated with the paranoid belief; see online Supplementary Table S2 for a full list of item pairs and analysis of their content).
Eigenvalue-analysis, parallel analysis and optimal coordinates of the Scree plot converged on a three-factor solution (see Fig. 1; detailed results regarding the Scree plot analyses are found in online Supplementary Table S3). The results of a subsequent promax-rotated EFA are shown in Table 2. As can be seen, most items showed large loadings (>0.600) on one factor without any substantial cross loadings. The three factors accounted for 68.0% of the total item variance (F1: 0.392, F2: 0.170, F3: 0.110) and showed medium to large intercorrelation (r F1,F2 = 0.56, r F1,F3 = −0.51, r F2,F3 = −0.44). Three items showed no loading above 0.600 and were excluded from further analyses. The remaining 25 items also showed sufficiently high communality (>0.5 for all items, see Table 2). Thus, the resulting first factor Persecutory Threat comprised of 15 items focusing on beliefs about intentional actions aimed at harming the respondent. The second factor, Paranoid Conspiracy, included six items that captured beliefs that COVID-19 is a means of people in power controlling the general population. Finally, the factor Interpersonal Mistrust (four items) captured the lack of trust that other people will behave responsibly during the pandemic. The subsequent CFA on the four remaining samples (combined) showed sufficient model fit of the three-factor solution for the 25-item PPS across all fit indices (χ2(272) = 2880, p < 0.001, CFI = 0.949, TLI = 0.943, RMSEA = 0.041, SRMR = 0.049, BIC = 97 978), with high standardised loadings (range: 0.699–0.914). The three factorial model showed better fit than a one-dimensional model χ2(275) = 11 485, p < 0.001, CFI = 0.742, TLI = 0.719, RMSEA = 0.091, SRMR = 0.115, BIC = 106 561; χ2-difference test: χ2(3) = 8604, p < 0.001).
Eigenvalue-analysis, parallel analysis and optimal coordinates for EFA.
Results of EFA (UK sample) and loadings from the CFA (USA, Germany, Australia and Hong Kong)
Note: Only loadings >0.100 are shown. Loadings printed in bold denote items included in the final version of the PPS.
a For a more precise estimation, the presented CFA loadings were estimated based on the full sample (UK, USA, Australia, Germany and Hong Kong). Factor correlations show the standardised linear correlations between the three factors in the EFA model (based on UK sample) and the CFA (based on combined USA, Australian, Germany and Hong Kong samples).
Measurement invariance
The results of the measurement invariance analyses are summarised in Table 3. As can be seen, the configural invariance model showed sufficient fit, indicating an equal factor structure across different samples and language versions. Differences in the fit-indices remained in an acceptable area and the BIC was comparatively low for weak invariance. For the strong invariance and the strict invariance models, however, ΔCFI exceeded threshold values and AIC and BIC were larger compared to the configural invariance model. Thus, the data suggested weak invariance. Based on this, we continued to use the PPS factor scores for the criterion validation (see Table 1, right column for the item loadings from the full sample CFA).
Results from the measurement invariance analyses
Note: ΔCFI/RMSEA/SRMR in bold indicate a difference in the respective fit index that exceeds the nested model-criteria for invariance.
General pandemic paranoia factor
To test whether the three PPS factors can be combined into one global indicator of pandemic paranoia, we repeated the CFA with an additional second-order factor. As can be seen in Fig. 2, all three factors showed large loadings on the second-order factor (Persecutory Threat: 0.899, Paranoid Conspiracy: 0.604 and Interpersonal Mistrust: 0.581).
PPS factor structure.
Since three first-order factors lead to a just-identified second-order factor model, we imposed further constraint to the second-order factor model in the form of equal loadings of all first-order factors on the second-order factor. The model showed a sufficient fit in the four site sample (χ2(274) = 2898, p < 0.001, CFI = 0.948, TLI = 0.943, RMSEA = 0.041, SRMR = 0.058). The pattern of results for the measurement invariance analysis using the full five-site sample remained unchanged, with the configural invariance model (CFI = 0.922, TLI = 0.915, RMSEA = 0.044, SRMR = 0.070, AIC = 119 292, BIC = 121 565) and the metric invariance model (CFI = 0.920, TLI = 0.918, RMSEA = 0.043, SRMR = 0.076, AIC = 119 519, BIC = 121 220) showing the sufficient fit, but no scalar invariance as indicated by a change in CFI above 0.01 (CFI = 0.901, TLI = 0.903, RMSEA = 0.047, SRMR = 0.078, AIC = 120 077, BIC = 121 289).
Internal consistency
Internal consistencies were good to excellent for Persecutory Threat (α = 0.97, ω = 0.98), Paranoid Conspiracy (α = 0.93, ω = 0.93) and Interpersonal Mistrust (α = 0.83, ω = 0.83) as well as the PPS Global score (α = 0.90, ω = 0.74).
Convergent validity
The results of the convergent validation analyses are summarised in Table 4. As can be seen, all three PPS factors showed the expected moderate to strong associations with a general tendency to experience paranoia (i.e. paranoid ideation as well as ideas of reference). Interpersonal Mistrust showed the expected moderate association with COVID-19 anxiety and a small to moderate association between the perceived risk and severity of an infection, whereas Persecutory Threat showed a moderate association with the perceived personal risk of infection, but consistently small correlations with COVID-19 anxiety and perceived infection severity. Paranoid Conspiracy showed no substantial correlation with COVID-19 anxiety or the perceived risk variables. There was, however, a strong correlation between this factor and general conspiracy mentality, whereas the other factors showed only small associations with this criterion. Interestingly, R-GPTS scores were more highly correlated with the PPS Global scores than factor scores.
Correlation between PPS factor scores and global scores and convergent validity criteria
Criterion validity
As a provisional examination of criterion validity, we compared scores across global and factor scores for those reporting a current mental health diagnosis (n = 528) v. not (n = 1982). Since there was no scalar invariance, we calculated the effect sizes per country and integrated them using a fixed effects model in order to achieve an unbiased estimate of the general effect size (based on Cohen's d; for the descriptive values and effect sizes per country, see online Supplementary Table S4). Across sites, PPS global sum scores were significantly higher in people self-reporting a current mental illness diagnosis (d = 0.173, Z = 3.38, p < 0.001, corresponding significant differences in four sites), as were Paranoid Conspiracy sum scores (d = 0.151, Z = 2.77, p = 0.006, corresponding significant differences in two sites) and Interpersonal Mistrust sum scores (d = 0.314, Z = 6.10, p < 0.001, corresponding significant differences in four sites). For Paranoid Threat, a minimal, non-significant average difference (d = 0.061, Z = 1.19, p = 0.234) emerged, but a pattern of significantly higher scores in people with a mental illness diagnosis was found in three of the five sites (see online Supplementary Table S4). Notably, the Australian sample showed some anomalous effects in that Paranoid Threat and Global scores were significantly higher in those not reporting an MH condition.
Test–retest reliability
In the UK subgroup (n = 98), the relationship between baseline PPS scores and 3-month scores were: Persecutory Threat [ICC = 0.75; 95% confidence interval (CI) 0.65–0.83], Paranoid Conspiracy (ICC = 0.64; 95% CI 0.50–0.75), Interpersonal Mistrust (ICC = 0.57; 95% CI 0.41–0.69) and PPS Global scores (ICC = 0.72; 95% CI 0.60–0.80).
Discussion
This paper reports on the development and validation of the PPS, a measure designed to assess paranoid thinking, from across the full spectrum of experience, and focusing specifically on the threat posed by others to oneself in the context of the pandemic. Factor analyses supported three highly related but distinguishable factors: Persecutory Threat, which converged most strongly with the R-GPTS, captured beliefs that others will intentionally cause one physical or psychological harm via the pandemic (e.g. deliberately infecting me, preventing me from protecting myself, spreading rumours I am infected, being hostile/critical towards me because they believe I am infected); Paranoid Conspiracy, which converged most strongly with the conspiracy mentality questionnaire, captured the belief that COVID-19 is a conspiracy generated by those in power (e.g. government, politicians) to threaten and/or control people and Interpersonal Mistrust, which converged most strongly with COVID-19 anxiety and perceived severity of illness if infected, captured concerns about not being able to trust others to follow government guidance and keep their communities safe. Internal consistency was good to excellent across global and factor scores. Criterion validity suggested the elevated levels of PPS global scores, interpersonal mistrust and paranoid conspiracy for participants who self-reported a current mental health diagnosis as compared to the those who did not. However, differences were not significant for persecutory threat. Anomalous findings arising for the Australian sample require further examination, including more fine-grained differentiation between mental disorders (i.e. psychotic v. non-psychotic disorders) in future studies.
Whilst the factors showed differential associations with other constructs, they nonetheless demonstrated large factor loadings onto a second-order factor model. Furthermore, the PPS global score was more strongly associated with persecutory beliefs than any individual factor. These data suggested that the factors assess distinguishable aspects of the overarching pandemic paranoia construct. The PPS global score may therefore most accurately capture the overall construct of pandemic paranoia, whereas the three first-order factors tap into interrelated facets of pandemic paranoia that crucially differ in the associated affective responses (e.g. the presence of fear).
Although many factors are likely to affect the experience of the pandemic across the sites here investigated (e.g. government, vaccine availability, health care systems, etc.), the measurement invariance data suggested a consistent factor structure across cultures and languages (English, German and Chinese). Based on the overall pattern of fit indices, however, using simple sum or factor scores to compare the absolute values across cultures/languages may be at risk for bias. The change in one answer category has the same meaning across sites, however the starting value for item ratings differed across sites (i.e. participants in one site tending to be more liberal for endorsing paranoid thoughts than participants from another site). In order to compare samples across countries/languages, we recommend – if possible – to calculate latent trait scores in a weak-invariance model as a starting point for cross-cultural analyses.
The findings should be considered in light of several limitations. Firstly, item generation was deductive rather than inductive; that is, PPS items were based on existing knowledge and measurement of paranoia in the general population, rather than through consultation with people experiencing paranoia during the pandemic. As such, aspects of pandemic-specific paranoia may have been missed and/or the existing items may have not optimally captured this construct. Inductive methods, such as quantitative methods, are well suited to generating new questionnaires; however, the time-sensitive nature of the pandemic prevented this. Relatedly, self-report measures of paranoia are unavoidably affected by measurement error. Triangulation methods, such as by assessing the convergence of PPS scores with, for example, clinician-based assessments, behavioural observations and/or information obtained from close others would be beneficial. Furthermore, consistent with several other paranoia scales, the PPS did not include reverse-coded items. Amongst other benefits, reverse coding helps to correct for acquiescence bias (Weijters & Baumgartner, Reference Weijters and Baumgartner2012). Correlating the PPS with other non-reverse coded questionnaires is therefore likely to include some degree of acquiescence bias. Secondly, whilst we purposefully sampled a representative sample from each region, there are groups that are nonetheless under-represented, such as those without access to a computer and those with low literacy skills. The sample was also limited to those who were aware of and signed up to Qualtrics, which may introduce unknown bias. Finally, in keeping with general paranoia measures such as the R-GPTS and the Paranoia Checklist, the PPS aimed to capture paranoid beliefs across the full spectrum of experience, thus including items that are less explicitly persecutory (e.g. items from Interpersonal Mistrust subscale). Consistent with the paranoia hierarchy model, our findings suggest that these milder beliefs are part of a broader paranoia spectrum, but future research understanding the link between milder and more sever beliefs is required.
Notwithstanding these limitations, the current findings have several implications. Our data suggest that within the umbrella term of pandemic paranoia, there are three clusters of concerns, each of which may have different implications. The content of the items measuring persecutory threat concerns appear to align most closely with paranoid beliefs seen in clinical groups. Future research would benefit from examining this further, such as by testing whether threat concerns exhibit characteristics that are reminiscent of clinical paranoia (e.g. distressing, preoccupying, persistent and adversely impacting functioning) and/or share common correlates with clinical paranoia (e.g. negative core beliefs, depression, anxiety, stress, victimisation, minority status, etc.). This factor could prove to be clinically useful, such as by helping to identify those with elevated and/or persistent threat concerns. Furthermore, if such threat concerns are reminiscent of clinical paranoia, it is possible that they could be effectively managed using evidence-based treatments for paranoia (e.g. Ellett et al., Reference Ellett, Tarant, Kouimtsidis, Kingston, Vivarelli, Mendis and Chadwick2020; Freeman et al., Reference Freeman, Dunn, Startup, Pugh, Cordwell, Mander and Kingdon2015). The second factor, which measured paranoid conspiracy concerns, may yield useful information for considering trust-building measures by governing authorities such as adopting transparency in communication with the general public and in decision-making processes. Our final factor, interpersonal mistrust, seems to relate to the trust that people have within their local communities for others to responsibly follow guidelines. This is likely to be more or less prevalent depending on how well governing authorities have the pandemic under control (e.g. current infection rates and vaccination rates) as well as the extent to which communities have united v. fractured in the context of the pandemic. It could be interesting to understand community-based factors that increase/decrease concerns about interpersonal mistrust, whether these concerns persist as the threat diminishes, and interventions that may help to reduce such concerns. Having established these factors, future research investigating the prevalence (Ellett et al., under review) and structure of pandemic paranoia is warranted.
In summary, preliminary examination of the psychometric properties of the PPS, across five international sites and three languages, using representative samples in terms of sex, age and educational attainment support the factorial stability, construct validity and internal reliability of the subscales. The PPS offers an exciting and unique opportunity to now examine pandemic paranoia more fully.
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/S0033291721004633
Financial support
This research received no specific grant from any funding agency, commercial or not-for-profit sectors.
Conflict of interest
None.
Ethical standards
The authors assert that all procedures contributing to this work comply with the ethical standards of the relevant national and institutional committees on human experimentation and with the Helsinki Declaration of 1975, as revised in 2008.
Open access
