Introduction
Breath carbon monoxide (CO) is a biochemical marker of recent cigarette smoking (see Bittoun, Reference Bittoun2008; Society for Research on Nicotine and Tobacco Subcommittee on Biochemical Verification, 2002). Breath CO correlates well with other established assays of cigarette smoking, such as COHb concentration in blood samples (Hald, Overgaard, & Grau, Reference Hald, Overgaard and Grau2003; Jarvis, Belcher, Vesey, & Hutchison, Reference Jarvis, Belcher, Vesey and Hutchison1986), and urinary cotinine levels (Marrone, Paulpillai, Evans, Singleton, & Heishman, Reference Marrone, Paulpillae, Evans, Singleton and Heishman2010; Marrone et al., Reference Marrone, Shakleya, Scheidweiler, Singleton, Huestis and Heishman2011), and has high levels of sensitivity and specificity in distinguishing between cigarette smokers and non-smokers (Javors, Hatch, & Lamb, Reference Javors, Hatch and Lamb2005; MacLaren et al., Reference MacLaren, Conigrave, Robertson, Ivers, Eades and Clough2010; Perkins, Karelitz, & Jao, Reference Perkins, Karelitz and Jao2013; Raiff, Faix, Turturici, & Dallery, Reference Raiff, Faix, Turturici and Dallery2010). Further, as the collection of blood or urine samples is relatively invasive, breath CO provides a useful assay for researchers, clinicians, and physicians as a non-invasive, efficient method for assessing recent cigarette smoking.
The clinical utility of breath CO has been established in a variety of contexts and across several populations of smokers. For example, adolescent (National Survey on Drug Use and Health, 2011) and pregnant (Cnattingius, Reference Cnattingius2004) smokers represent vital target populations for smoking prevention and cessation efforts. Given the often surreptitious nature of cigarette smoking in these populations, breath CO provides an accurate and reliable method for verifying self-report assessments, which may be unreliable under some conditions (Dolcini, Adler, Lee, & Bauman, Reference Dolcini, Adler, Lee and Bauman2003; Shipton et al., Reference Shipton, Tappin, Vadiveloo, Crossley, Aitken and Chalmers2009). Breath CO has also been used as a sensitive measure for the verification of smoking abstinence in the context of behavioural interventions for smoking cessation (i.e., contingency management (CM); e.g., Stitzer & Bigelow, 1982). Some CM studies have shown that lower breath CO measures are associated with decreases in negative health outcomes. For example, Heil et al. (Reference Heil, Higgins, Bernstein, Solomon, Rogers, Thomas and Lynch2008) found that lower breath CO measures were associated with beneficial effects on foetal weight and femur length in a sample of pregnant smokers. In addition, Dallery and colleagues (Dallery & Glenn, Reference Dallery and Glenn2005; Dallery, Glenn, & Raiff, Reference Dallery, Glenn and Raiff2007; Stoops et al., Reference Stoops, Dallery, Fields, Nuzzo, Schoenberg, Martin and Wong2009) have examined the delivery of CM for smoking cessation through a novel Internet-based platform. Their research indicates that breath CO is a feasible and acceptable method for the assessment of smoking status without requiring participants to visit a research or clinical setting (see Dallery & Raiff, Reference Dallery and Raiff2011). Lastly, recent evidence has found that simply monitoring breath CO levels can result in reduced levels of smoking (Beard & West, Reference Beard and West2012). These data point to the need for breath CO meters that are accessible outside of clinical and academic research settings.
Breath CO meters that are often used in research and clinical settings provide quantitative results regarding CO levels in parts-per-million (ppm). This precision is beneficial from a research and clinical perspective because it provides an assessment of smoking status beyond merely discriminating between smokers and non-smokers. Such precision, however, also causes these models to be relatively expensive, making them inaccessible to some individuals and institutions that could benefit from these instruments (e.g., schools, midwives, pregnant smokers, etc.).
Recognizing this barrier, manufacturers have introduced models that are substantially less expensive. Although this reduction in price could allow researchers and health professionals with limited financial resources to access this technology, this cost-reduction is largely accomplished by reducing the precision of the instrument. For example, breath CO meters that provide semi-quantitative results are commercially available. Instead of providing a quantitative measure of breath CO in ppm, semi-quantitative meters provide measures that indicate the ppm range in which an individual's breath CO level falls.
The purpose of the current research was to examine the sensitivity and specificity of a lower-cost semi-quantitative CO meter relative to that of a quantitative meter, the sensitivity and specificity of which has been established empirically (MacLaren et al., Reference MacLaren, Conigrave, Robertson, Ivers, Eades and Clough2010; Raiff et al., Reference Raiff, Faix, Turturici and Dallery2010). Specifically, using self-reported smoking status to classify the results of each breath CO measure (i.e., true-positive, false-positive, etc.), we assessed sensitivity and specificity across a range of CO abstinence cutoff criteria for both the semi-quantitative and quantitative meters, and compared the accuracy of the two meters statistically.
Methods
Participants
Adult smokers and non-smokers between 18 and 65 years of age were recruited either through the University of Florida, Department of Psychology's undergraduate research participant pool or through in-person solicitation in the Gainesville, Florida metropolitan area. Gainesville is a medium-sized city (~125,000 citizens) located in north-central Florida and is surrounded by a rural landscape. Participants were divided into three different groups based on their self-reported smoking status: non-smokers (never smoked cigarettes or other substances; n = 50), light smokers (1–10 cigarettes/day; n = 40), and regular smokers (11+ cigarettes/day; n = 40). The University of Florida Institutional Review Board approved all procedures.
Materials
Breath samples were collected using both the piCO+ and babyCOmpact Smokerlyzer® CO meters (Bedfont Scientific USA, Williamsburg, VA). The babyCOmpact provides semi-quantitative results using seven pairs of LEDs. More specifically, each pair of LEDs represents a different Level that corresponds to a specific range in ppm (Level 1 – 0–3 ppm; Level 2 – 4–6 ppm; Level 3 – 720 ppm; Level 4 – 21–25 ppm; Level 5 – 26–35 ppm; Level 6 – 36–50 ppm; Level 7 – greater than 51 ppm). Similar to the babyCOmpact, the piCO+ meter provides feedback regarding CO levels with twin LEDs. However, the piCO+ also presents CO results in a numeric format on a liquid crystal display in 1 ppm increments within a range of 0–100 ppm.
The piCO+ meters were initially calibrated by coVita™ (Haddonfield, NJ), the official US distributor of Bedfont Smokerlyzer™ products. Consistent with the manufacturer's recommendations, piCO+ meters were recalibrated at least once every six months. The babyCOmpact meters were calibrated by the manufacturer and cannot be recalibrated. To ensure accuracy, the babyCOmpact and piCO+ meters were periodically tested (~monthly) using a control gas with a CO concentration of 20 ppm. Digital stopwatches were used to time the duration of exhalation for each breath sample and the 2 minute interval between samples (Raiff et al., Reference Raiff, Faix, Turturici and Dallery2010). Research assistants started the timer when participants placed their lips onto the CO meter mouthpiece and stopped the timer when participants removed their lips from the mouthpiece.
Procedures
Assessments
After obtaining informed consent, participants completed a questionnaire packet including questions about demographics, current smoking status (e.g., time since last cigarette, number of cigarettes smoked/day, etc.) and smoking history.
Experimental Conditions
During each session, participants provided four breath samples: two samples each from a piCO+ meter and a babyCOmpact meter. The sequence in which participants provided the four breath samples was randomized across participants. Before providing a breath sample, a member of the research team instructed participants about the experimental procedure and modelled the correct way to exhale into the CO meters. Participants then completed the experimental procedure. For each sample, participants were asked to take a deep breath and hold it for 15 seconds. Once the participant exhaled into the meter, the CO result and duration of exhalation were recorded, and the stopwatch was reset and restarted to time the 2 minute interval that was required to elapse before the collection of the following sample. At the end of the session, participants either received course credit or a $5 Walmart gift card.
Data Analysis
Factorial analyses of variance (ANOVAs) were conducted to assess differences in CO levels and duration of exhalation between the three smoking groups and across the four samples. Bonferroni post-hoc analyses were conducted when main effects were observed. Independent-samples t-tests were conducted to assess differences in the number of cigarettes smoked/day, and time since last cigarette. Results of the analyses were deemed statistically significant at p < .05. Analyses were conducted using SPSS Statistics, Version 21.0 (IBM® Corporation, Armonk, NY).
Receiver-operating characteristic (ROC) curves were calculated to assess the sensitivity and specificity of the piCO+ and the babyCOmpact at different CO abstinence-cutoff values (Javors et al., Reference Javors, Hatch and Lamb2005). In this context, sensitivity and specificity refer to the ability of the instrument to accurately identify smokers and non-smokers, respectively. Specifically, to generate these curves, the rate of true-positive results (i.e., sensitivity) was plotted against the rate of false-positive results (i.e., 100 – specificity). Sensitivity was calculated by assessing the percentage of CO samples provided by smokers that were positive (i.e., above the CO cutoff value), across a range of CO abstinence-cutoff criteria. Specificity was calculated by assessing the percentage of CO samples provided by non-smokers that were negative (i.e., at or below the CO cutoff value), across a range of CO abstinence-cutoff criteria. Area-under-the-curve (AUC) and standard error were estimated for each plot and compared using a non-parametric method to assess differences in two or more dependent ROC curves (Delong, Delong, & Clarke-Pearson, Reference DeLong, DeLong and Clarke-Pearson1988). Self-reported smoking status was used as the standard to classify breath CO measures (i.e. true-positive, false-positive, etc.). The ROC analysis was conducted using MedCalc Statistical Software, Version 12.7.2.0 (MedCalc Software, Ostend, Belgium).
Results
Participant Characteristics
Demographic and smoking characteristics of participants are shown in Table 1. As expected, light and regular smokers differed on the number of cigarettes smoked per day [t(70) = 8.16, p < .001]. As shown in Table 1, participants in the regular smoker group smoked more cigarettes than participants in the light smoker group. No significant differences in time since last cigarette were found between smoking groups.
Participants’ Demographic and Smoking Characteristics
Note. Values are means ± S.D. unless otherwise indicated.
Zero non-smokers, six light smokers, and eight regular smokers reported recently smoking something other than a cigarette. These participants reported that it had been 65 minutes and 129 minutes (light smokers and regular smokers, respectively), since smoking this other substance. In addition, six non-smokers, nine light smokers, and 16 regular smokers reported being exposed to other sources of smoke recently. Of these participants, all six of the non-smokers, six light smokers, and all 16 of the regular smokers reported being exposed to second-hand smoke. The other three light smokers reported being exposed to marijuana smoke, car exhaust, and burning wood, respectively.
CO Values and Duration of Exhalation
The three smoking groups differed significantly for both piCO+ [F(2, 127) = 88.79, p < .001] and babyCOmpact CO readings [F(2, 127) = 107.53, p < .001]. Post-hoc analyses found that non-smokers provided lower piCO+ and babyCOmpact readings (M ppm = 3.3 ± 7.90; M Level = 1.1 ± .82) than light smokers (M ppm = 23.6 ± 7.88; M Level = 3.7 ± .82) who provided lower readings on both meters than regular smokers (M ppm = 34.1 ± 7.88; M Level = 4.5 ± .82). Means of the two within-subject CO measures did not significantly differ for the piCO+ (M ppm: 20.3 vs. 20.4) or the babyCOmpact (M Level: 3.0 vs. 2.9).
The three smoking groups also differed significantly in duration of exhalation [F(2, 127) = 8.0, p = .001]. Post-hoc analyses found that non-smokers exhaled for a shorter average duration (M s = 8.56 ± 3.62) than both light smokers (M s = 11.78 ± 3.62) and regular smokers (M s = 12.61 ± 3.62) who did not differ from one another on this measure. Further, within-subject analysis found that duration of exhalation across the four CO measures also significantly differed [F(3, 381) = 9.5, p < .001]. Post-hoc analyses found that duration of exhalation during the two piCO+ measurements (11.68 ± 7.21s vs. 10.96 ± 5.96 s), the two babyCOmpact measurements (10.51 ± 5.13 s vs. 10.02 ± 4.91 s), and both the first pair and second pair of the piCO+ and babyCOmpact measurements, respectively, significantly differed.
Sensitivity and Specificity
Sensitivity and specificity proportions for the piCO+ (≤ 1–10 ppm) and the babyCOmpact (≤ Levels 1–3) are shown in Table 2. For the piCO+, a cutoff of ≤ 7 ppm maximised the likelihood of accurately classifying smokers and non-smokers (sensitivity + specificity = 1.88). Using this criterion as a cutoff, five measures (5%) collected from four non-smokers (8%) were incorrectly categorised as positive, and ten measures (13%) collected from seven light smokers (18%) were incorrectly categorised as negative. For the babyCOmpact, a cutoff of ≤ Level 2 (4–6 ppm) maximised the likelihood of accurately classifying smokers and non–smokers (1.87). Using this Level as a cutoff, all measures collected from nonsmokers were correctly categorised as negative, whereas 15 measures (19%) collected from nine light smokers (23%) and five measures (6%) collected from three regular smokers (3%) were incorrectly categorised as negative.
Sensitivity and Specificity of the piCO+ and babyCOmpact Meters
As shown in Figure 1, the areas under the ROC curves for the piCO+ and babyCOmpact, respectively, were 98.4% (SE = 0.7%) and 96.0% (SE = 1.1%). The ROC analysis indicated that the difference between the two AUCs was statistically significant (p < .05).
The percentage of false-positive results (100-specificity) was plotted against the percentage of true-positive results (sensitivity) across the entire range of breath CO measures obtained for the piCO+ and babyCOmpact meters.
Discussion
The findings from the current study suggest a semi-quantitative meter can distinguish between smokers and non-smokers. Although the babyCOmpact was slightly less accurate than the piCO+, this level of accuracy is commensurate with previous research examining CO cutoffs with more precise breath CO meters (Javors et al., Reference Javors, Hatch and Lamb2005; MacLaren et al., Reference MacLaren, Conigrave, Robertson, Ivers, Eades and Clough2010; Perkins et al., Reference Perkins, Karelitz and Jao2013; Raiff et al., Reference Raiff, Faix, Turturici and Dallery2010). Using self-reported smoking status as the standard to classify measures, a cutoff of ≤ Level 2, which maximised sensitivity and specificity, resulted in no false-positive and 16% false-negative results. Interestingly, analysis of the piCO+ measures in the current study revealed that a cutoff of ≤ 7 ppm maximised sensitivity and specificity – a cutoff higher than recent research examining the accuracy of the piCO+ meter which recommends an abstinence cutoff of ≤ 4 ppm (MacLaren et al., Reference MacLaren, Conigrave, Robertson, Ivers, Eades and Clough2010; Raiff et al., Reference Raiff, Faix, Turturici and Dallery2010).
The disparity in the sensitivity and specificity calculations of the present study and those of previous research was largely a function of relatively elevated CO measures provided by non-smokers. Although the average CO measure collected from non-smokers in the current study (3.3 ppm) fell below this recommended criterion of smoking abstinence, out of 100 piCO+ measures, 21 measures collected from 12 non-smokers were higher than four ppm, 12 measures collected from seven non-smokers were higher than five ppm, 10 measures collected from five non-smokers were higher than six ppm, and six measures collected from four non-smokers were higher than seven ppm. Of the 12 non-smokers who provided a piCO+ measure above 4 ppm, zero reported smoking anything else (e.g., marijuana) that day, current use of other tobacco products (i.e., cigars), or living with a smoker, and only two reported recent exposure to second-hand smoke.
Another factor that could potentially explain these elevated measures is duration of exhalation (Raiff et al., Reference Raiff, Faix, Turturici and Dallery2010). Although the average duration for the piCO+ measures > 4 ppm collected from non-smokers was lower than the average duration of the remaining piCO+ measures collected from non-smokers (8.21 sec vs. 9.28 sec, respectively), previous research suggests a direct relationship between speed of exhalation and CO measures (Raiff et al., Reference Raiff, Faix, Turturici and Dallery2010). In short, of the variables that have been found to affect CO measures that were assessed in the current study (i.e., recent exposure to sources of smoke, physical exertion, faulty or poorly calibrated meter), none were systematically related to these elevated measures.
Additionally, several factors related to the expiration and/or endogenous production of CO could have contributed to these elevated measures. For example, lactose intolerance (McNeill, Owen, Belcher, Sutherland, & Fleming, Reference McNeill, Owen, Belcher, Sutherland and Fleming1990) as well as several clinical disorders including types 1 and 2 diabetes, asthma, chronic obstructive pulmonary disorder, and bronchiectasis have all been found to increase expired breath CO levels (Underner & Peiffer, Reference Underner and Peiffer2010). Unfortunately, these variables were not assessed in the current study.
The CO measures of this subset of non-smokers heavily influenced specificity calculations for both the piCO+ and the babyCOmpact. Of the 12 non-smokers who provided a piCO+ measure above 4 ppm, six provided a corresponding babyCOmpact measure of Level 2. Only one non-smoker provided a babyCOmpact measure of Level 2 and a corresponding piCO+ measure of ≤ 4ppm. As shown in Table 2, for the piCO+, relative to a cutoff of ≤ 7 ppm, specificity was 15% lower using a cutoff of ≤ 4 ppm. For the babyCOmpact, relative to a cutoff of ≤ Level 2, specificity was 8% lower using a cutoff of ≤ Level 1. Additionally, the method used to calculate sensitivity and specificity may also slightly understate the sensitivity of the babyCOmpact. For example, when calculating these proportions for the babyCOmpact, using a ≤ Level 2 cutoff, a reading of Level 1 or 2 for a self-reported smoker was considered a false-negative. However, breath CO has a short half-life (3–6 hours; Benowitz et al., 2002) and therefore is only sensitive to recent smoking. Of the 11 smokers who provided a babyCOmpact reading at Levels 1 or 2, four reported not smoking a cigarette within 4 hours.
In summary, these results suggest that a semiquantitative meter can accurately distinguish smokers from non-smokers. The average price of commercially available meters with higher CO measurement resolution is ~$975 (US dollars) – a cost that is likely prohibitive to low-income smokers and the agencies (e.g., community health clinics) that provide many of these individuals with primary-care services (Shin, Rosenbaum, & Paradise, 2012). The results of the current study are encouraging and provide support for a semi-quantitative approach to the analysis of breath CO as a viable, low-cost method of verifying smoking status for smoking-cessation researchers and clinicians, as well as primary-care physicians and other healthcare professionals.
Acknowledgements
We thank Alessandra Troncoso and Pamela Carey for assisting with data collection.
Financial support
This work was supported by the Department of Psychology at the University of Florida.
Conflict of interest
None.
Ethical standards
The authors assert that all procedures contributing to this work comply with the ethical standards of the relevant national and institutional committees on human experimentation and with the Helsinki Declaration of 1975, as revised in 2008.
