INTRODUCTION
There is compelling evidence supporting the implementation of peer-review methods in radiation oncology. Research has shown that peer review is of critical importance and has the potential to improve both quality and safety in radiation oncology.Reference Marks, Adams and Pawlicki1–Reference Lawrence, Whiton and Symon3 Peer-review programmes at every stage in radiation oncology management can potentially eliminate some of the treatment inaccuracies that presumably result from poor management decisions, variations in treatment protocols and lack of experience and robust quality assurance programmes. In addition, peer review can identify trends and barriers associated with quality radiotherapy and share the best practice or recommend changes accordingly.
In a recent document released by the American Society for Radiation Oncology (ASTRO), peer review was identified as a critical component of a radiation oncology quality assurance programme, which can be used to ensure safety in all the processes involved.Reference Lawrence, Whiton and Symon3 Peer review can also ensure that right decisions are made and there is consistency of practice.Reference Pawlicki and Mundt4 However, the concern in various radiotherapy departments is the development of a peer-review method that is secure and effective in addressing the shortfalls in quality assurance programmes.Reference Marks, Adams and Pawlicki1 Smaller centres may have only one radiation oncologist, and thus face-to-face peer review is not feasible. Web-based techniques can allow non-competing centres to collaborate and address safety and quality issues in radiation oncology.
Unlike face-to-face methods, web-based peer reviews may raise concerns such as effectiveness in addressing quality and safety issues data, security and confidentiality among other technical and logistical concerns. With such potential problems, the acceptance of web-based peer review in radiation oncology may require a transformational process. One example of web-based peer-review programme includes the chartrounds.com,5 which has brought together radiation oncologists and physicists to connect and discuss cancer-management issues. Nevertheless, with the ever-changing treatment protocols and regimens for different types of cancers, the need for physician-to-physician support is paramount. Evidence on the feasibility of such peer reviews is crucial in stimulating engagement into web-based peer reviews.
This study analysed an inter-centre, case-oriented and web-based peer-review programme against defined safety and quality targets. In addition, it analysed the experiences of radiation oncology professionals who attended the web-based peer-review meetings in order to identify priorities for improvement.
BACKGROUND
The term peer review has been used in radiation oncology to encompass a multitude of activities including chart rounds, multidisciplinary meetings, physics audits and ‘physician-to-physician’ peer reviews. Peer review can be defined as a collaboration between two or more individuals for an extended period, with regular meetings and activities (at least once a month), in order to improve quality and safety.Reference Grol6 A variety of subjects, interventions and methods are used in a planned and structured manner. The process may include setting criteria, data collection, performance appraisal, exchange of experiences, developing guidelines, solving problems in practice and making specific arrangements for achieving changes. Collaboration with respected peers and honest mutual provision and acceptance of evaluation and support are central to the process of peer review. Richard GrolReference Grol6 described peer review as ‘a continuous, systematic and critical reflection by a number of care providers on their own and colleagues’ performance using structured procedures with the aim of achieving continuous improvement of the quality of care.’
Marks et al.Reference Marks, Adams and Pawlicki1 identified several clinical situations where peer review is anticipated to be useful in radiation oncology. One practical example is when a patient presents with a recurrence for re-treatment. The paucity of research regarding re-treatments necessitates collaboration with more experienced peers to come up with the best treatment approach. In addition, the delivery of radiotherapy continues to become sophisticated, promising increased accuracy for targeting malignancies and avoiding normal tissues, with technologies such as stereotactic treatments and image-guided radiotherapy.Reference Bissonnette, Balter and Dong7 These evolutions of radiotherapy practices require stringent measures to ensure both quality and safety in healthcare provision. Although recognised standards of care such as the ASTRO have advocated for peer reviews in radiation oncology, there is a paucity of data regarding the development and evaluation of radiation oncology in both ‘traditional’ and web-based peer-reviews programmes in light of technological and intellectual developments.
To achieve high standards of quality and safety healthcare, ongoing practice evaluation of the processes that ensure that such standards are achieved may be necessary. Most organisations measure quality of clinical practice against identified performance benchmarks by peers, professional organisations or national regulatory bodies. KleineReference Kleine8 identifies ‘benchmarking’ as an excellent tool used to identify priorities for improvement, identify partners who have accomplished certain goals and identify suitable radiotherapy practices. As a continuous improvement tool, benchmarking fits within the conceptual framework of Deming’s wheel of quality.Reference Amina, Levif and Michael9
Despite the availability of benchmarking tools, determining that appropriate standards are being met can be challenging and may depend on the goals of the peer-review programme. In radiation oncology, certain targets should be a priority in peer-review presentations. For example, the inclusion of the prescribed dose and discussion of the dose-volume constraints (DVCs) are crucial. To assist the implementation of peer-review programmes, ASTRO provided a comprehensive list of potential targets and the need for prioritisation.Reference Marks, Adams and Pawlicki1 This list includes physician-, physics- and radiation therapists-focused tasks. For example, the physician-focused tasks outlined included the decision to treat; planning directive or goals (e.g., dose-volume constraints, goals for normal tissues and target(s), prescribed doses and fractionation); clinical plan quality (e.g., achieved dose-volumes); technical plan quality (completeness, complexity, as good as reasonably achievable, acceptable to meet the prescription intent); and a planned method for setup verification (e.g., imaging). These targets can also be used to ensure that all important aspects are prioritised and discussed in the peer-review meeting. However, the ability to visualise treatment images requires that technical aspects of the meeting be addressed carefully. In web-based peer reviews, the suitability of audio and visual tools designed to assist users among other concerns such as data security may also be evaluated.Reference Marks, Adams and Pawlicki1 This may elevate safety and quality standards in web-based peer reviews and promote engagement between radiation oncology professionals.
In response to the need for radiation oncologists’ collaboration, a web-based peer-review programme was developed between three Caribbean radiation oncology centres in October 2011. An online meeting is conducted on weekly basis to discuss all definitive-treatment cases before treatment. Radiation oncologists, physicists and dosimetrists attended the meetings to discuss patient management plans. Dose prescriptions and DVCs for the treatment plans were reviewed with the aid of a commercial meeting software – GoToMeeting’ (Citrix Systems Inc, Fort Lauderdale, FL, USA).Reference Briggs10
MATERIALS AND METHODS
Analysis of the peer-review meetings
The researchers participated in all the meetings and evaluated the case presentations based on ten targets listed on the evaluation form (Table 1). Four categories of cancer patients (breast, prostate, cervical and head and neck) were evaluated. These categories represented commonly reviewed cases treated with curative intent. Each week, patients were selected at random from the three participating centres. A total of 40 patients were evaluated (10 from each category) between July and November 2013. The evaluation form was designed with reference to comprehensive physician-focused ASTRO potential targets for radiation oncology peer review.Reference Marks, Adams and Pawlicki1 To achieve comparability, a scoring system was used and the results were analysed.
Peer-review content analysis form
Survey
A questionnaire was administered to all the regular attendees of the peer-review meetings. This comprised four radiation oncologists, four medical physicists and two medical dosimetrists from three radiation oncology centres. The rationale of the survey was to analyse the experiences of the multidisciplinary team and also to identify priorities for improving the peer-review programme. An e-mail outlining the purpose, time commitments and anticipated outcomes for enrolment in the survey was sent to all the participants of the weekly peer-review meeting. Survey questions were both quantitative and qualitative in nature. The survey conducted had both closed and open-ended questions in order to allow the participants to comment on the peer-review method.
RESULTS
Evaluation of the peer-review meetings
The results from the evaluation of the peer-review meetings were stratified according to the diagnosis categories. The meetings presented scored an average of 8·5 on a scale of 10 points. The head and neck cancer cases achieved a mean score of 8·73 (SD=0·9), followed by the breast cancer with a mean score of 8·63(SD=0·92). The reviews for prostate cancer patients scored a mean of 8·45(SD=1·02), whereas the reviews for cervical cancer patients scored the lowest (M=8·27, SD=1·5).
In addition to evaluating the content of the presentations, the researchers examined whether the present peer review had some measures of security and confidentiality in place. One drawback regarding this aspect was the non-availability of standards from the radiation oncology community to evaluate against. As a commercial softwareReference Briggs10 was used for the meetings, it provided features that addressed security and confidentiality issues as shown in Table 2. It is important to recognise that these were security measures claimed by the commercial software.
Minimum Security and confidentiality provided by the commercial software
Abbreviation(s): SSL, Secure Sockets Layer; are cryptographic protocols designed to provide communication security over a computer network.
Survey results
Qualitative analysis of the study consisted of using content analysis procedures. The researchers analysed the open-ended responses and identified themes that could be used for improving the peer-review programme. Quantitative analysis of the study consisted of using the Likert scale technique11 to score the participant’s response to each statement. Each statement was individually scored and then all the statements were summed to get representation of the participants’ perceptions. This scoring was based on higher points for positive responses and lower points for negative responses. For example, the rating scale was strongly agree (4), agree (3), disagree (2), strongly disagree (1) and not applicable (0).
In section 1, one question addressed the perceptions of participants towards the role of peer review in promoting practice advancement. The response was very favourable (M=3·44, SD=0·53). The survey also demonstrated that approximately all participants had a positive perception towards the role of peer review in promoting safety and quality in radiation oncology. Three specific questions directly addressed the role of peer review in motivation for facilitating self-regulation of practice, research interest and increasing accountability. Tables 3–5 show the overall perception to these sub-questions, respectively.
Facilitates self-regulation of radiation oncology practice
Motivates research interest
Increases accountability
Section 2 of the questionnaire focused on the feedback from peers. The results were positive with regard to the feedback from colleagues during the peer-review meetings. However, there were contrasting opinions on what could be done to improve the peer-review meeting. For instance, one participant recommended the discussion of the dosimetry and physics aspects of the presented treatment plans, whereas another participant preferred the discussion to be limited to content that facilitated clinical decision-making only. There were differences in opinion regarding time allocation for the peer-review meeting and the time of the day that the peer-review should be conducted. Some participants recommended the discussion of only 1 or 2 cases per session in depth; likewise, another participant recommended only the discussion of non-standard treatment cases. However, 89% of the participants did not give an opinion on the number of cases that should be reviewed per session.
In section 3, the participants were asked to rate the quality of the audio–visual aspect of the web-based meeting, and the results are shown in Table 6.
The quality of the audio and visual aspects
The rationale was to get feedback on the quality of the isodose plans and dose-volume histograms (DVHs) presented in the meetings. The participants were satisfied with the web-based peer review, except for one recommendation to increase the size of the DVH to improve clarity. Another recommendation was that presenters should take time in presenting each case so that the audience can see the isodose distribution and DVH clearly and be able to give appropriate feedback.
DISCUSSION
The peer-review meetings
During the development of a peer-review programme, radiation oncology professionals define a set of targets that should be covered in the peer-review discussions. The results from the analysis of the peer-review meetings were stratified according to the diagnosis categories and were rated. In general, it seemed that more complex cases were able to engage and stimulate longer discussions.
Technical requirements
Web-based peer-review programmes require suitable technology that facilitates discussion of treatment plans and DVHs with clarity. Implementing peer review requires research and planning.Reference Likert12 The authors analysed the technical and logistical concerns highlighted by the participants in this study. This study shows that the majority of participants were satisfied with clarity of the audio–visuals during the web-based presentations. Based on the experience from the web-based peer-review meetings, sometimes Internet and hardware failures can affect the flow of reviews. Therefore, there is need to ensure usage of good computers and networking systems. The literature highlights the need for improving technologies for radiation oncology peer reviews. Palta et al.Reference Palta, Frouhar and Dempsy13 presented an infrastructure of comprehensive tools that could be used for web-based peer review through use of the resource centre for emerging technologies system. This system allows submission, auto-archiving, web-based reviews through retrieval and evaluation of diagnostic images and treatment planning data. However, its implementation in small radiotherapy centres could be limited by lack of knowledge about its availability, lack of information technology skills and cost to purchase the required software. Therefore, there still is a need for affordable, effective and secure models for web-based peer reviews.
Feedback and recording of outcomes.
Michael et al.Reference Michael, Sophie and McGowan14 highlighted the benefits of recording the outcomes as part of the treatment record. In addition, they addressed the patterns of recording peer-review outcomes. It is important that in web-based peer-review development users decide ‘what’, ‘how’ and ‘where’ to record the outcomes and feedback from the meetings. Alhough our survey results reflect a positive attitude towards the feedback from colleagues, in the present study, the authors did not analyse how the participating centres recorded the outcomes and feedback from the reviews.
In the literature, some barriers to honest feedback include fear of retaliation that may tarnish the notion of peer review. Haag-heitman and Vicki GeorgeReference George and Haag-Heittman15 highlighted the lack of constructive feedback and giving inflated affirming feedback, ineffective ‘pal’ review, as a contributor to the negative connotation that professionals may have about peer-review processes. La LopaReference La Lopa16 refers to the ‘halo effect bias’ arising from a reviewer who has positive feelings for their reviewee, and thus provides more favourable feedback than their performance would merit. During the early stages of implementation of peer review, there is might be a fear of offending the other colleagues, but the trust and collegiality tend to improve with time.
Despite such negative connotations, Chamunyonga and BridgeReference Chamunyonga and Bridge17 argue that peer reviews bring many benefits that transfer well to radiotherapy practice. Brooks et al.Reference Brooks, Olsen, Rieger-Kligys and Mooney18 described the development of a peer-review process to enhance professional practice. Additional benefits include the ‘affirmation and inspiration’ gained by the peers, which increases confidence about aspects of their practice.Reference Gusic, Hageman and Zenni19 However, for peer review to be successful, an open and honest partnership must be established between peers. Evidence suggests that pairing for mutual review nurtures the provision of practice sharing and support.Reference Hansen20 Thus, it is important that partnerships are formed from radiation oncology centres, which are unlikely to be in competition with each other.
Security and confidentiality
The exposure of patient information over the internet requires strict guidance to ensure that the risk of security breach is avoided.Reference Gritzalis21 The present analysis indicates some level of security and confidentiality through the use of login passwords – the desirable security features that include ‘end-to-end’ security – over Secure Sockets Layer-protected connections such as those claimed by commercially available software such as GotoMeeting (Citrix Solutions Inc, USA).11
The use of online services for peer review in radiation oncology requires that the users identify the safety features necessary to prevent any potential threats to confidentiality. There may be a need to decide on which features to use so that confidentiality issues are addressed. Although inter-centre peer review has been advocated by ASTRO,Reference Marks, Adams and Pawlicki1 there are no data available on inter-country peer reviews in radiation oncology. As additional evidences on web-based peer reviews increase, professional bodies and accrediting organisations may need to develop minimum security and confidentially standards for web-based peer reviews.
Limitations
A limitation to the methodology was the small sample size due to the small number of participants. Although the population size was small, their responses were useful, as they allowed the participating centres to review the peer-review method based on their recommendations.
Recommendations
The following recommendations can be made based on the results of this study;
1. Small radiation oncology centres should consider collaborating with nearby centres to conduct peer-review case conferences.
2. Appropriate technology is required to ensure secure platforms, good audio clarity of the presenters and the audience and quality images to view anatomical detail.
3. To successfully implement web-based peer reviews, there is need for structures and procedures for standardisation and recording of feedback and outcomes.
Conclusions
Peer review is a method of improving professional growth and quality of care. It can identify trends, challenges and barriers to safe delivery of high quality radiotherapy and recommend appropriate changes. The results of this study suggest that simple inter-centre web-based peer-review case conferences are feasible and offer an alternative technique for peer review in radiation oncology. A willingness to participate, coupled with the right tools for a structured peer-review programme, can allow radiation oncologists to collaborate from different locations. Although limitations such as the need for data security and confidentiality exist, they can be overcome by the use of appropriate technology provided by commercially available software.
Acknowledgements
The author acknowledges his colleagues at the Cancer Centre Bahamas and Southern Medical Clinic for their encouragement and advice.
Ethical Standards
Ethics approval was sought and obtained from the author’s institutional review board.
Funding
No source of funding was considered.
Conflicts of Interest
None.
