Introduction
Mailed questionnaires are commonly used in quality control assessment of nasal surgery.Reference Rhee, Sullivan, Frank, Kimbell and Garcia1 Response rates vary, and ratings from non-responders may therefore have a significant impact on the results. We have previously published a prospective study using telephone interviews of the non-responders to assess the influence of their ratings on the results of surgery.Reference Egeland, Tarangen, Gay, Døsen and Haye2 The mail responders had greater improvement in obstruction scores than telephone interviewees (non-responders).
We are not aware of any other study on the results of nasal septoplasty that has focused on the ratings of non-responders. Therefore, we performed a new prospective study using a new patient cohort and another interviewer to further assess non-response. We wanted to compare these two studies to confirm or disprove the earlier findings, and to assess the influence of the interviewer on the results.
In order to facilitate comparison of the studies, we have included the data from the earlier published study as cohort one. Permission was given by the main author and the Editor of the Journal of Laryngology and Otology. The new study is referred to as cohort two.
Materials and methods
Both studies were approved by the ethics committee of Lovisenberg Diaconal Hospital, Oslo, Norway, and include patients who underwent septoplasty with or without turbinoplasty at Lovisenberg Diaconal Hospital. Patients were at least 16 years of age and had not had any other concomitant nasal or sinus surgery, or any nasal or sinus disease except allergy.
We routinely use the Nasal Surgical QuestionnaireReference Haye, Tarangen, Shiryaeva and Døsen3 for assessing the result of nasal septal surgery. The pre-operative version is completed in the morning on the day of surgery. The questionnaire contains a visual analogue scale (VAS) for scoring nasal obstruction during the day. The VAS has a 10 cm line, with the left end representing no obstruction and the right end representing complete obstruction. Patients are asked to rate their sense of nasal obstruction on the scale with a vertical line. The score is measured as the distance in millimetres from the left-hand side of the scale to the marked line.
For other nasal symptoms and therapies, four-point Likert scales are used with the following response categories: none, mild, moderate, and either severe symptoms, daily symptoms or daily use of nasal medication. In addition, questions about smoking habits and allergies are included. The post-operative version of the Nasal Surgical Questionnaire has an added question about the retrospective rating of improvement in nasal obstruction: ‘Is your breathing now completely, much, somewhat improved, unchanged or worse?’
The Nasal Surgical Questionnaire was mailed to each patient five and a half months post-surgery together with a cover letter signed by a surgeon at the department and a prepaid return envelope. The mailing was suspended during the summer holiday season from mid-June and resumed in mid-August, which increased the response interval for some patients to seven and a half months. This summer hiatus coincided with the main pollen season.
Three weeks later a reminder with the same questionnaire was mailed to those who had not returned their questionnaire. Patients who did not return either questionnaire (mail non-responders) were contacted and interviewed by telephone from 8 to 13 months post-surgery. Telephone call attempts were limited to a maximum of three per patient. The telephone interviews took place outside of the pollen seasons. The patients were asked to respond to all items on the Nasal Surgical Questionnaire, indicating how they felt on a normal day without any infection. In the telephone interviews, the patients were asked to rate their sense of nasal obstruction on a numeric rating scale ranging from 0 to 10 (0 = no obstruction, and 10 = complete obstruction) in lieu of the VAS.
Cohort one included patients who were operated on from April to October 2014, with a female nurse performing all of the telephone interviews. Cohort two included patients who were operated on from November 2014 to December 2015, with a male surgeon performing all the telephone interviews. Neither of the interviewers was known to the patients.
Statistical analysis
All analyses were conducted using SPSS® (version 24) statistical software. Descriptive statistics (means, standard deviations and frequencies) were used to summarise sample characteristics and questionnaire responses.
Independent sample t-tests and Mann–Whitney U tests were used to compare patients who completed the mailed questionnaire (mail responders) with those who completed the telephone interview (mail non-responders) on continuous variables. Chi-square tests (or Fisher's exact test when any expected cell frequencies were less than 5) were used for group comparisons on categorical variables. The Wilcoxon signed rank test was used to assess the difference between pre-operative and post-operative VAS scores in each group (mail and telephone). Marginal homogeneity tests were applied to assess the difference between pre-operative and post-operative symptoms, and use of nasal medication. A significance level of p < 0.05 was used for all analyses.
Results
Cohort one (patients operated on between April and October 2014) consisted of 182 patients, of whom 122 (67.0 per cent) returned the mailed questionnaire. Of the remaining 60 patients, 58 (96.7 per cent) completed a telephone interview.
Cohort two (patients operated on between November 2014 and December 2015) consisted of 454 patients, of whom 294 (64.8 per cent) returned the mailed questionnaire. Of the remaining 160 patients, 135 (84.4 per cent) completed a telephone interview.
In both studies, the mean interval between surgery and the return of the mailed Nasal Surgical Questionnaire was 6.5 months (range: 6–8 months), and the mean interval between surgery and the telephone interview was 10 months (range: 8–13 months).
The mean age of the mail responders in both studies was significantly higher than of the non-responders, but there was no significant gender difference between the groups (Table 1).
Mean age and gender distribution in mail responders and mail non-responders
* P-value for t-test (age) and Pearson chi-square test (gender) comparing mail responders and non-responders. SD = standard deviation
The changes in VAS obstruction scores among mail responders and in numeric rating scale obstruction scores among telephone responders for both studies are described in Table 2. In both studies, there was significantly more improvement in the obstruction scores for mail responders than for telephone responders. There was also a small improvement in other nasal symptoms and slightly less use of nasal medication after surgery in both studies, but no significant differences between the mail and telephone responders (data not shown).
Obstruction scores for mail responders (VAS) and telephone responders (numeric rating scale)
* P-value for Mann–Whitney U test comparing mail responders and telephone responders. VAS = visual analogue scale; SD = standard deviation
The results from the retrospective ratings of obstruction improvement in both studies are presented in Table 3. Patients in the telephone interview (mail non-responders) group reported significantly better results than the patients in the mail responder group in both studies.
Retrospective rating of improvement in nasal obstruction after septoplasty
Comparison of patients completing the questionnaire by mail or by telephone.
* P-value for Pearson chi-square test comparing mail and telephone responders
The mean improvement in VAS and numeric rating scale obstruction scores was not significantly different between smokers and non-smokers (mail responders: p = 0.578; telephone interviewees: p = 0.543) or between allergic and non-allergic patients (mail responders: p = 0.242; telephone interviewees: p = 0.085).
Cohorts one and two comparison
There was no significant difference in gender or age between the two different cohorts (age: p = 0.509; gender: p = 0.572) or within the mail responder (p = 0.493, p = 0.682) and telephone interview (p = 0.952, p = 0.366) subgroups.
Both studies showed significantly more improvement in the VAS and numeric rating scale obstruction scores after surgery in the mail responder group than in the mail non-responder (telephone interview) group. There was no significant difference between the two studies in the mean improvement in VAS and numeric rating scale obstruction ratings for either the mail (p = 0.426) or telephone (p = 0.095) subgroup.
The retrospective subjective rating of improvement in obstruction was not significantly different between the two studies among the mail respondents (p = 0.065). However, in the telephone interview group, the retrospective rating of improvement was significantly better when the interview was conducted by the surgeon than when conducted by the nurse (p < 0.001).
Discussion
In both studies, the pre-operative VAS and numeric rating scale obstruction scores were higher and the post-operative scores were lower among mail responders compared with mail non-responders (telephone interviewees). As a result, the improvement in VAS and numeric rating scale scores was significantly better in the mail respondents compared with the telephone interviewees.
Telephone interviewees in both studies recorded significantly better retrospective ratings of improvement in nasal obstruction than mail respondents did. This difference was even more pronounced in cohort two, in which the telephone interview was performed by the male surgeon. The better improvement ratings obtained in the telephone interviews compared with the mailed questionnaires is probably because of the personal contact with the patient during the interview. Such positive and socially desirable responses in interviews are well-described in other studies.Reference Bowling4 We tried to reduce this influence by using interviewers who were unknown to the patients and who had no prior knowledge of the patients or their case history. Relative anonymity may reduce socially acceptable responses,Reference Haye, Døsen, Tarangen and Shiryaeva5 but given that we did not conceal the interviewer's professional status from the patients, our efforts may have been insufficient. An effect of the gender of the interviewer could not be explored in this study.
The patients scored their post-operative VAS and numeric rating scale scores without knowledge of their pre-operative scores. The retrospective ratings, however, were dependent on the patients’ memory of their pre-operative sense of obstruction. This may explain the seemingly contradictory finding that telephone responders had more improvement according to the retrospective ratings, but mail responders had more improvement when using prospective VAS and numeric rating scale scores, which might be less vulnerable to social desirability bias. Therefore, we consider prospective scorings more reliable than retrospective ones.
The two study cohorts were obtained from the patient population in two different years. Ideally, the patients should have been randomised to either cohort to control for any potential effects of time. However, there were no statistically significant differences in demographic data between them. The improvement in VAS and numeric rating scale obstruction scores was not different between smokers and non-smokers or between allergic and non-allergic patients in either cohort. Therefore, we find the studies comparable.
The difference in VAS and numeric rating scale score improvement between mail and telephone responders might be due to the difference in rating scales between the mail responders using VAS and the telephone interviewees using numeric rating scale. We have, however, published a study showing robust correlations between VAS and numeric rating scale ratings of nasal obstruction.Reference Bijur, Latimer and Gallagher6 Such good correlations between VASs and numeric rating scales are consistent with similar studies of pain ratings.Reference Hjermstad, Fayers, Haugen, Caraceni, Hanks and Loge7, Reference Stewart, Smith, Weaver, Witsell, Yueh and Hannley8 We are therefore confident that the VAS and numeric rating scale ratings are comparable.
The time-lag of about three and a half months between the mail responses and the telephone interviews may have influenced the results. However, studies of the results of nasal surgery have shown that the improvement in nasal obstruction after nasal surgery is not only sustained from 3 to 6 monthsReference Hong, Lee, Cho, Jang, Jung and Kim9–Reference Hsu, Tan, Chang, Chu, Chiu and Pan11 but also from 6 to 12 months.Reference Toyserkani and Frish12 These results indicate that the time lag in our study is unlikely to have influenced the results.
• Mail respondents experienced more improvement in nasal obstruction after nasal septoplasty than non-respondents when prospective ratings were employed
• Low response rates may cause an overestimation of the results of nasal surgery
• Mail non-responders (telephone interviewees) provided better improvement ratings than mail responders using retrospective scoring
• Retrospective ratings are dependent on the patients’ memory, and telephone interviewees tend to give more socially desirable answers
• Prospective scoring systems are therefore preferable to retrospective ones
The protocols of both studies were designed to reduce confounding factors, such as infection and allergy. The patients were asked to reply based on how they felt on a normal day without any infection. Most of our operations, mailing of questionnaires and telephone interviews were performed outside of the main pollen seasons. We believe that these measures resulted in the lack of significant differences in improvement scores between allergic and non-allergic patients.
In both studies, the mail non-responders (telephone interviewees) were younger than patients who responded to the mailed questionnaire. This finding is consistent with some prior studies of septoplastyReference Croy, Hummel, Pade and Pade13, Reference Bulut, Wallner, Plinkert, Prochnow, Kuhnt and Baumann14 but not with others.Reference Dinis and Haider15–Reference Hing, Smith, Hooper, Song and Donell18 There was no difference in gender between the mail responders and non-responders, which is consistent with some prior studies,Reference Dinis and Haider15–Reference Hing, Smith, Hooper, Song and Donell18 although other studies have reported a higher response rate among females.Reference Croy, Hummel, Pade and Pade13, Reference Bulut, Wallner, Plinkert, Prochnow, Kuhnt and Baumann14 The lack of demographic differences between our two cohorts suggests that the sample demographics are unlikely to explain the observed differences between the cohorts among the telephone responders.
We used telephone interviews instead of recalling the patients for interviews to gain access to the non-responders. Recalling, consulting and interviewing that many patients would be costly and time consuming. The attendance rate would probably be very low, and the interviews of patients at the clinic would have been subjected to even greater interviewer influence. We believe that a telephone interview is the best way to obtain results from mail non-responders.
A strength of the two studies is that they were prospective and performed at one clinic with a standard protocol. The use of different interviewers allowed us to assess the influence that the interviewer may have on the results. A weakness of the studies is that the post-operative recordings had to be obtained using two different methods (mail and telephone), which are known to influence patients’ responses.
Acknowledgements
We want to thank Laila Randen Stenberg for help with the questionnaires.
Competing interests
None declared
