Hostname: page-component-7b7567c8c-tnzcf Total loading time: 0 Render date: 2026-11-12T20:06:37.625Z Has data issue: false hasContentIssue false
  • English
  • Français

Which Mindfulness Measures To Choose To Use?

Published online by Cambridge University Press:  17 December 2015

Yuanmei (Elly) Qu,
Marie T. Dasborough and
Gergana Todorova
Yuanmei (Elly) Qu
Affiliation:
Department of Management, School of Business, University of Miami
Marie T. Dasborough*
Affiliation:
Department of Management, School of Business, University of Miami
Gergana Todorova
Affiliation:
Department of Management, School of Business, University of Miami
*
Correspondence concerning this article should be addressed to Marie T. Dasborough, Department of Management, School of Business, University of Miami, 5250 University Drive, 414F Jenkins Building, Coral Gables, FL 33146. E-mail: m.dasborough@miami.edu
Rights & Permissions [Opens in a new window]

Extract

Hyland, Lee, and Mills (2015) specified the two most popular scales for mindfulness: the Freiburg Mindfulness Inventory (FMI; 30 items, Buchheld, Grossman, & Walach, 2001; 14 items, Walach, Buchheld, Buttenmüller, Kleinknecht, & Schmidt, 2006) and the Mindfulness Attention and Awareness Scale (MAAS; Brown & Ryan, 2003). However, the popularity of these measures does not necessarily mean that they are of high quality. Especially considering the complex epistemology and ontology of the mindfulness construct, we should apply mindfulness assessments with caution. More specially, according to item development theorists in the industrial and organizational (I-O) area (see Hinkin, 1998; Hinkin & Tracey, 1999; Kerlinger & Lee, 2000; Schriesheim, Powers, Scandura, Gardiner, & Lankau, 1993), scholars must select measures that have (a) a clear operational definition; (b) alignment between definition and measure (content validity); (c) high reliability and (d) high construct validity; and (e) high criterion-related validity. However, it is not clear which of the available mindfulness assessments satisfies these criteria and to what extent. In this commentary, we assess currently used measures based on these criteria and provide directions for future research.

Information

Type
Commentaries
Information
Industrial and Organizational Psychology , Volume 8 , Issue 4 , December 2015, pp. 710 - 723
Copyright
Copyright © Society for Industrial and Organizational Psychology 2015 

Hyland, Lee, and Mills (Reference Hyland, Lee and Mills2015) specified the two most popular scales for mindfulness: the Freiburg Mindfulness Inventory (FMI; 30 items, Buchheld, Grossman, & Walach, Reference Buchheld, Grossman and Walach2001; 14 items, Walach, Buchheld, Buttenmüller, Kleinknecht, & Schmidt, Reference Walach, Buchheld, Buttenmüller, Kleinknecht and Schmidt2006) and the Mindfulness Attention and Awareness Scale (MAAS; Brown & Ryan, Reference Brown and Ryan2003). However, the popularity of these measures does not necessarily mean that they are of high quality. Especially considering the complex epistemology and ontology of the mindfulness construct, we should apply mindfulness assessments with caution. More specially, according to item development theorists in the industrial and organizational (I-O) area (see Hinkin, Reference Hinkin1998; Hinkin & Tracey, Reference Hinkin and Tracey1999; Kerlinger & Lee, Reference Kerlinger and Lee2000; Schriesheim, Powers, Scandura, Gardiner, & Lankau, Reference Schriesheim, Powers, Scandura, Gardiner and Lankau1993), scholars must select measures that have (a) a clear operational definition; (b) alignment between definition and measure (content validity); (c) high reliability and (d) high construct validity; and (e) high criterion-related validity. However, it is not clear which of the available mindfulness assessments satisfies these criteria and to what extent. In this commentary, we assess currently used measures based on these criteria and provide directions for future research.

There have been 11 mindfulness measures published to date, 3 of which we excluded here because of their infrequent use (citations in Google Scholar less than 30), including the Development Mindfulness Survey (Solloway & Fisher, Reference Solloway and Fisher2007), the Effects of Meditation Scale (Reavley & Pallant, Reference Reavley and Pallant2009), and the Langer Mindfulness/Mindlessness Scale (Haigh, Moore, Kashdan, & Fresco, Reference Haigh, Moore, Kashdan and Fresco2011). Hence in this commentary, we focus on the eight measures presented in Table 1: MAAS, FMI (14 items),Footnote 1 the Kentucky Inventory of Mindfulness Skills (KIMS; Baer, Smith, & Allen, Reference Baer, Smith and Allen2004), the Toronto Mindfulness Scale (TMS; Lau et al., Reference Lau, Bishop, Segal, Buis, Anderson, Carlson and Devins2006), the Five Facets Mindfulness Questionnaire (FFMQ; Baer, Smith, Hopkins, Krietemeyer, & Toney, Reference Baer, Smith, Hopkins, Krietemeyer and Toney2006), the revised Cognitive and Affective Mindfulness Scale (CAMS; Feldman, Hayes, Kumar, Greeson, & Laurenceau, Reference Feldman, Hayes, Kumar, Greeson and Laurenceau2007), the Southampton Mindfulness Questionnaire (SMQ; Chadwick et al., Reference Chadwick, Hember, Symes, Peters, Kuipers and Dagnan2008), and the Philadelphia Mindfulness Scale (PHLMS; Cardaciotto, Herbert, Forman, Moitra, & Farrow, Reference Cardaciotto, Herbert, Forman, Moitra and Farrow2008). We first elaborate on our specified standards, then apply these standards to assess established mindfulness measures, and finally provide our evaluation results. Table 2 summarizes the results of our assessment.

Table 1.

Mindfulness Measures

Table 2.

Evaluation of Mindfulness Measures

Note. EFA = exploratory factor analyses; CFA = confirmatory factor analyses.

a Chadwick et al. (Reference Chadwick, Hember, Symes, Peters, Kuipers and Dagnan2008) did not provide reliability of each dimension, and thus we could not evaluate their reliability level.

1. Clear Operational Definition

Without operational definitions, it is not possible for scholars to develop measures for empirical studies (Hinkin, Reference Hinkin1998; Schwab, Reference Schwab, Cummings and Staw1980). A good operational definition of mindfulness should be able to reflect the complex mindful state and efficiently direct instrument development (Chiesa, Reference Chiesa2013). As we have learned from definitions of other constructs (e.g., transformational leadership, Van Knippenberg & Sitkin, Reference Van Knippenberg and Sitkin2013), a good definition should (a) state clear definitions for the construct and each dimension (if multidimensional), (b) articulate relationships among distinct dimensions, and (c) specify inclusion and exclusion criteria for the definitions.

A Clear Definition for Mindfulness and Each Dimension

Despite some overlaps, different terminology has been used to define the construct of mindfulness (Dimidjian & Linehan, Reference Dimidjian and Linehan2003), and there have been incongruent arguments regarding the dimensions. Whereas Brown and Ryan (Reference Brown and Ryan2003) referred to mindfulness as a one-dimensional construct, Bishop et al. (Reference Bishop, Lau, Shapiro, Carlson, Anderson, Carmody and Devins2004) described mindfulness as a two-dimensional construct, comprising self-regulation of attention and orientation to experience. Other scholars promote even more dimensions, such as Baer et al. (Reference Baer, Smith, Hopkins, Krietemeyer and Toney2006), who include observing, describing, acting with awareness, nonreactivity to inner experience, and nonjudging of inner experience. We do not advocate any particular number of dimensions but rather call for them to be clearly stated in an operational sense, so they may be used to guide empirical research. By illustration, Brown and Ryan (Reference Brown and Ryan2003; MAAS) referred to mindfulness as a one-dimensional construct and clearly defined it as “the state of being attentive to and aware of what is taking place in the present” (p. 822). Cardaciotto et al. (Reference Cardaciotto, Herbert, Forman, Moitra and Farrow2008; PHLMS) also provided clear definitions for mindfulness and both of their proposed dimensions: present-moment awareness and acceptance. As such, we rate both MAAS and PHLMS as high on this criterion (see Table 2). Baer et al. (Reference Baer, Smith, Hopkins, Krietemeyer and Toney2006; FFMQ), however, failed to provide operational definitions for each dimension prior to developing their instrument.

Articulation of the Relationships Among Distinct Dimensions

As demonstrated by Chiesa (Reference Chiesa2013), definitions for distinct components are not enough for a clear operational definition, because scholars might not be able to capture the synergistic or mutually reinforcing interrelationships among distinct components. As such, consistent with what we have learned from the transformational leadership construct (Van Knippenberg & Sitkin, Reference Van Knippenberg and Sitkin2013), articulating the relationships among components becomes essential. For all six multidimensional measures we examined, only Cardaciotto et al. (Reference Cardaciotto, Herbert, Forman, Moitra and Farrow2008; PHLMS) specified the relationship that is proposed to exist between the two dimensions of mindfulness. Therefore, we encourage further research into the relationships between the mindfulness dimensions.

Criteria for Inclusion and Exclusion of Mindfulness Definition

Following the suggestions of Van Knippenberg and Sitkin (Reference Van Knippenberg and Sitkin2013), clear inclusion and exclusion criteria are essential for researchers to clarify a boundary for a construct in general and its components in particular. With that said, mindfulness scholars should specify why they choose particular dimensions instead of others. More important, this boundary should allow researchers to separate the mindfulness construct from closely related constructs, such as emotional intelligence.

Most scholars used previous definitions for their inclusion criteria. For example, Feldman et al. (Reference Feldman, Hayes, Kumar, Greeson and Laurenceau2007) specified four dimensions: attention, present focus, awareness, and acceptance for the CAMS measure, based on previously published definitions. Exclusion of other dimensions, on the other hand, was mostly based on the rule that any new dimension cannot be inferred from or implied by established dimensions. For example, Brown and Ryan (Reference Brown and Ryan2003) argued that acceptance should not be added as a unique dimension because it is so closely tied to awareness, and separately it offered no additional explanatory power. Cardaciotto et al. (Reference Cardaciotto, Herbert, Forman, Moitra and Farrow2008), however, argued that the degree to which there was mutual influence between awareness and acceptance was an open question; hence, acceptance should be incorporated as another dimension. We consider both studies to have done a good job in drawing on theoretical evidence to justify their inclusion and exclusion decisions. Other studies lacked these necessary justifications; for example, Baer et al. (Reference Baer, Smith and Allen2004; KIMS) and Baer et al. (Reference Baer, Smith, Hopkins, Krietemeyer and Toney2006; FFMQ) failed to adequately provide inclusion and exclusion criteria.

2. Alignment Between Definition and Measure (Content Validity)

Content validation is critical in connecting definitions with corresponding measures (Kerlinger & Lee, Reference Kerlinger and Lee2000). Schriesheim et al. (Reference Schriesheim, Powers, Scandura, Gardiner and Lankau1993) argued that high content validity should be guaranteed in the first place for item development procedures, because otherwise scholars might end up producing consistent measures (reliable) that describe a totally different construct from that which was intended to be measured (low content validity). A disconnect between mindfulness and the items used to measure it could explain inconsistent or unexpected findings in relation to the nomological network of mindfulness.

We evaluate the quality of content validity based on standards specified by Hinkin (Reference Hinkin1998), Hinkin and Tracey (Reference Hinkin and Tracey1999), and Schriesheim et al. (Reference Schriesheim, Powers, Scandura, Gardiner and Lankau1993). To develop measures with high content validity, scholars use either a deductive approach or a combination of inductive and deductive approaches to generate items, ask experts to evaluate item descriptions according to operational definitions, conduct exploratory factor analyses (EFAs), and accordingly revise their items. Then confirmatory factor analyses (CFAs) using a new sample are recommended to make sure that the new measure fits the theoretical definitions.

Among the eight measures for mindfulness we evaluate here, only MAAS and PHLMS followed the above procedures and produced new items with high content validity. FMI, SMQ, and FFMQ, however, ignored previous definitions (if any), and items were proposed without effective references to definitions. Although TMS intended to follow the operational definition of Bishop et al. (Reference Bishop, Lau, Shapiro, Carlson, Anderson, Carmody and Devins2004), it ended up being mostly data driven: The dimensions were altered based on the EFA results. On the contrary, realizing the danger of being guided by EFA results, Feldman et al. (Reference Feldman, Hayes, Kumar, Greeson and Laurenceau2007) made the opposite mistake by choosing not to use EFA to revise items in their CAMS instrument, and therefore their items ended up having low factor loadings.

3. Reliability

Reliability refers to the precision or accuracy of a measure, and it is a necessary (but not sufficient) condition for construct validity (Hinkin, Reference Hinkin1998). Hence, reliability is essential to ensure high measurement quality (Kerlinger & Lee, Reference Kerlinger and Lee2000). Nunnally (Reference Nunnally1978) noted that a measure with reliability higher than .70 was acceptable. According to our summary, all measures of mindfulness, except for the CAMS (low reliability) and the SMQ (reliability of each dimension was missing), held reliability (Cronbach's alpha) of each dimension higher than .72 (the nonreactivity to inner experience dimension of the FFMQ in a student sample is an exception; it had a reliability of 0.67). We therefore consider this criterion to be the strongest across the board in our evaluation of published mindfulness measures.

4. Construct Validity

High construct validity is important in guaranteeing the correspondence between the latent constructs and the specific measures (Kerlinger & Lee, Reference Kerlinger and Lee2000). With high construct validity, we are more confident that our measures have high overlap with alternative measures and are distinguished from measures of different constructs. That is, construct validity includes convergent validity (e.g., high correlations between measures of similar constructs) and discriminant validity (e.g., low correlations between measures of dissimilar constructs; Campbell & Fiske, Reference Campbell and Fiske1959).

Five (MAAS, KIMS, FFMQ, CAMS, and PHLMS) out of eight mindfulness measures met the standards specified above. FMI and TMS failed to conduct convergent validity tests. SMQ, on the other hand, did not specify its discriminant validity. Overall, construct validity was not highly problematic for most of the mindfulness scales.

5. Criterion-Related Validity

By putting the focal construct in a nomological network and calculating the correlations or incremental relationships between the measure of the focal construct and the measures of other constructs, scholars are able to infer whether the measure accurately reflects the focal construct (Cronbach & Meehl, Reference Cronbach and Meehl1955). Criterion-related validity is designed to fulfill this purpose. According to Kerlinger and Lee (Reference Kerlinger and Lee2000), criterion-related validity incorporates concurrent validity (e.g., relationships between mindfulness and related outcome variables measured at the same time) and predictive validity (e.g., relationships between mindfulness and related outcome variables measured sequentially).

Among all measures, MAAS demonstrated expected relationships with related outcome variables, including well-being, both for concurrent validity and predictive validity. FMI, KIMS, CAMS, and PHLMS, however, failed to conduct concurrent validity and predictive validity analyses, and thus from those scale development articles, we were not able to detect their quality of concurrent validity or predictive validity. Although TMS developers did not conduct concurrent validity analyses, they conducted mindfulness interventions and found support for expected relationships between this measure and outcome variables, including subjective stress and psychopathology.

FFMQ and SMQ, on the other hand, only employed concurrent validity tests for examining its nomological network. Specifically, FFMQ did a better job than the SMQ, in that the regression analyses enabled Baer et al. (Reference Baer, Smith and Allen2004) to conclude incremental relationships while controlling other variables, but correlation analyses as adopted by Chadwick et al. (Reference Chadwick, Hember, Symes, Peters, Kuipers and Dagnan2008) could not achieve this function. In sum, establishing concurrent and predictive validity (or a clear discussion of these procedures) was missing for all measures, except for the MAAS, which established both types of validity.

So Which Measures To Select?

Using the five general standards specified above, we evaluated the eight most common mindfulness measures. We summarized our evaluation reasons and results in Table 2. More specifically, for each standard, when measures achieved at least two third of our criteria, we rated them as high; if measures met less than a third of all criteria we specified, we assessed them as low; for those measures in between, we labeled them as moderate.

Considering the missing evidence for criterion-related validity, we first evaluated all measures together based on the first four standards and found that MAAS showed the highest quality, with PHLMS showing the second highest, and KIMS showing the third. We do not recommend the other five measures because they hold low quality in at least one standard. Specifically, the low quality on operational definitions and content validity criteria is difficult to remedy, because these standards must be met before choosing the final version of the measurement instrument. As such, we are reluctant to suggest KIMS, considering that its operational definition neither failed to clarify inclusion and exclusion criteria nor specified the relationships among distinct components.

MAAS demonstrates a clear operational definition, has evidence of high reliability and validity, and thus is a highly recommended measure to use. Although original development of PHLMS did not satisfy the fifth criterion, we conducted further examination of this measure. To assess concurrent validity and predictive validity for PHLMS, we sought other articles that described the nomological network for mindfulness using PHLMS. We found that PHLMS demonstrated high concurrent validity in Ruocco and Direkoglu (Reference Ruocco and Direkoglu2013) and moderate predictive power in Butryn et al. (Reference Butryn, Juarascio, Shaw, Kerrigan, Clark, O'Planick and Forman2013). Therefore, we recommend scholars apply either MAAS or PHLMS in their studies.

Conclusions

Incongruous mindfulness definitions and measures indicate the ambiguity and inaccuracy of established mindfulness research and, more important, require scholars to wisely select measures. Given the plentitude of mindfulness measures and the epistemological and ontological complexity of the construct, we scrutinized and rated eight widely used mindfulness measures, guided by standards specified by item development theorists in the I-O area. Most common problems were related to the lack of a clear mindfulness definition that establishes dimensions and relationships between dimensions, as well as missing criterion-related validity assessment. Our evaluations specified MAAS and PHLMS as the two most desirable (valid and reliable) measures currently available. We urge scholars to cautiously select mindfulness measures. We also encourage scholars to move the field forward by establishing a common and clear operational definition.

Footnotes

1 We choose to comment on FMI (14 items) rather than FMI (30 items), because FMI (30 items) was designed for participants with meditation experience only.

References

Baer, R. A., Smith, G. T., & Allen, K. B. (2004). Assessment of mindfulness by self-report the Kentucky Inventory of Mindfulness Skills. Assessment, 11 (3), 191–206.CrossRefGoogle ScholarPubMed
Baer, R. A., Smith, G. T., Hopkins, J., Krietemeyer, J., & Toney, L. (2006). Using self-report assessment methods to explore facets of mindfulness. Assessment, 13 (1), 27–45.Google Scholar
Bishop, S. R., Lau, M., Shapiro, S., Carlson, L., Anderson, N. D., Carmody, J., . . . , Devins, G. (2004). Mindfulness: A proposed operational definition. Clinical Psychology: Science and Practice, 11 (3), 230–241.Google Scholar
Brown, K. W., & Ryan, R. M. (2003). The benefits of being present: Mindfulness and its role in psychological well-being. Journal of Personality and Social Psychology, 84 (4), 822–848.Google Scholar
Buchheld, N., Grossman, P., & Walach, H. (2001). Measuring mindfulness in insight meditation (Vipassana) and meditation-based psychotherapy: The development of the Freiburg Mindfulness Inventory (FMI). Journal for Meditation and Meditation Research, 1 (1), 11–34.Google Scholar
Butryn, M. L., Juarascio, A., Shaw, J., Kerrigan, S. G., Clark, V., O'Planick, A., & Forman, E. M. (2013). Mindfulness and its relationship with eating disorders symptomatology in women receiving residential treatment. Eating Behaviors, 14 (1), 13–16.CrossRefGoogle ScholarPubMed
Campbell, D. T., & Fiske, D. W. (1959). Convergent and discriminant validation by the multitrait–multimethod matrix. Psychological Bulletin, 56 (2), 81–105.Google Scholar
Cardaciotto, L., Herbert, J. D., Forman, E. M., Moitra, E., & Farrow, V. (2008). The assessment of present-moment awareness and acceptance the Philadelphia Mindfulness Scale. Assessment, 15 (2), 204–223.Google Scholar
Chadwick, P., Hember, M., Symes, J., Peters, E., Kuipers, E., & Dagnan, D. (2008). Responding mindfully to unpleasant thoughts and images: Reliability and validity of the Southampton Mindfulness Questionnaire (SMQ). British Journal of Clinical Psychology, 47 (4), 451–455.Google Scholar
Chiesa, A. (2013). The difficulty of defining mindfulness: Current thought and critical issues. Mindfulness, 4 (3), 255–268.Google Scholar
Cronbach, L. J., & Meehl, P. E. (1955). Construct validity in psychological tests. Psychological Bulletin, 52 (4), 281–302.CrossRefGoogle ScholarPubMed
Dimidjian, S., & Linehan, M. M. (2003). Defining an agenda for future research on the clinical application of mindfulness practice. Clinical Psychology: Science and Practice, 10 (2), 166–171.Google Scholar
Feldman, G., Hayes, A., Kumar, S., Greeson, J., & Laurenceau, J.-P. (2007). Mindfulness and emotion regulation: The development and initial validation of the Cognitive and Affective Mindfulness Scale-Revised (CAMS-R). Journal of Psychopathology and Behavioral Assessment, 29 (3), 177–190.Google Scholar
Haigh, E. A., Moore, M. T., Kashdan, T. B., & Fresco, D. M. (2011). Examination of the factor structure and concurrent validity of the Langer Mindfulness/Mindlessness Scale. Assessment, 18 (1), 11–26.Google Scholar
Hinkin, T. R. (1998). A brief tutorial on the development of measures for use in survey questionnaires. Organizational Research Methods, 1 (1), 104–121.Google Scholar
Hinkin, T. R., & Tracey, J. B. (1999). An analysis of variance approach to content validation. Organizational Research Methods, 2 (2), 175–186.Google Scholar
Hyland, P. K., Lee, R. A., & Mills, M. J. (2015). Mindfulness at work: A new approach to improving individual and organizational performance. Industrial and Organizational Psychology: Perspectives on Science and Practice, 8 (4), 576–602.Google Scholar
Kabat-Zinn, J. (2003). Mindfulness‐based interventions in context: Past, present, and future. Clinical Psychology: Science and Practice, 10 (2), 144–156.Google Scholar
Kerlinger, F. N., & Lee, H. B. (2000). Foundations of behavioral research. Fort Worth, TX: Harcourt College.Google Scholar
Lau, M. A., Bishop, S. R., Segal, Z. V., Buis, T., Anderson, N. D., Carlson, L., & Devins, G. (2006). The Toronto Mindfulness Scale: Development and validation. Journal of Clinical Psychology, 62 (12), 1445–1467.CrossRefGoogle ScholarPubMed
Nunnally, J. C. (1978). Psychometric theory. New York, NY: McGraw-Hill.Google Scholar
Reavley, N., & Pallant, J. F. (2009). Development of a scale to assess the meditation experience. Personality and Individual Differences, 47 (6), 547–552.CrossRefGoogle Scholar
Ruocco, A. C., & Direkoglu, E. (2013). Delineating the contributions of sustained attention and working memory to individual differences in mindfulness. Personality and Individual Differences, 54 (2), 226–230.CrossRefGoogle Scholar
Schriesheim, C. A., Powers, K. J., Scandura, T. A., Gardiner, C. C., & Lankau, M. J. (1993). Improving construct measurement in management research: Comments and a quantitative approach for assessing the theoretical content adequacy of paper-and-pencil survey-type instruments. Journal of Management, 19 (2), 385–417.Google Scholar
Schwab, D. P. (1980). Construct validity in organizational behavior. In Cummings, L. L. & Staw, B. M. (Eds.), Research in organizational behavior (Vol. 2, pp. 3–43). Greenwich, CT: JAI Press.Google Scholar
Solloway, S. G., & Fisher, W. P. (2007). Mindfulness practice: A Rasch variable construct innovation. Journal of Applied Measurement, 8 (4), 359–372.Google Scholar
Van Knippenberg, D., & Sitkin, S. B. (2013). A critical assessment of charismatic-transformational leadership research: Back to the drawing board? The Academy of Management Annals, 7 (1)1–60.Google Scholar
Walach, H., Buchheld, N., Buttenmüller, V., Kleinknecht, N., & Schmidt, S. (2006). Measuring mindfulness—The Freiburg Mindfulness Inventory (FMI). Personality and Individual Differences, 40 (8), 1543–1555.Google Scholar
Figure 0

Table 1.Mindfulness Measures

Figure 1

Table 2.Evaluation of Mindfulness Measures