Introduction
The delivery of radiation therapy has become more accurate with the introduction of image guidance and adaptive radiation therapy techniques, which promise improved accuracy of tumour targeting and avoidance of normal tissues.Reference Lausch, Chen, Ward, Gaede, Lee and Wong 1 , Reference Lim, Bezjak and Higgins 2 The rapid changes in treatment techniques and the introduction of technologies such as Tomotherapy, CyberKnife and Gamma Knife, and the use of Stereotactic Body Radiation Therapy (SBRT) or stereotactic radiosurgery techniques has also added to the complexity of radiation treatment planning requiring advanced critical knowledge and skills. Moreover, the use of reduced margins in these techniques means that robust measures to ensure accuracy and precision needed to be introduced.Reference Dutta, Balaji, Murli, Sudahar, Gopalakrishna and Potharaju 3 Peer review of treatment plans has emerged as an opportunity to improve plan consistency, decrease the need for replanning and improve quality of care through the safe delivery of high-quality radiotherapy plans.Reference Adams, Marks, Pawlicki, Hayman and Church 4 – Reference Boxer, Forstner and Kneebone 6
The concept of peer review as a quality assurance (QA) tool in clinical radiation oncology is not new, evidence shows that peer review has the potential to improve both quality and safety in radiation oncology.Reference Marks, Adams and Pawlicki 5 – Reference Brundage, Foxcroft, McGowan, Gutierrez, Sharpe and Warde 7 In addition, literature shows how peer review programmes can eliminate treatment inaccuracies that can result from poor management decisions, variations in treatment protocols and lack of experience.Reference Boxer, Forstner and Kneebone 6 Although organisations such as the American Society for Radiation Oncology identified peer review as a crucial component of a radiation oncology QA programme;Reference Marks, Adams and Pawlicki 5 several authors have identified challenges and barriers in its use in routine clinical practice.Reference Marks, Adams and Pawlicki 5 , Reference Brundage, Foxcroft, McGowan, Gutierrez, Sharpe and Warde 7 , Reference Chamunyonga and Bridge 8 Nonetheless, recent studies have revealed encouraging benefits that have rekindled the interest in radiation oncology peer review, particularly the review of treatment plans.Reference Matuszak, Hadley and Feng 9 , Reference Lo, Liu and Chan 10
Despite the growing consensus on the benefits of peer review, there is a paucity of anecdotal and empirical data on the use of either systematic or individual peer review of treatment plans by radiation therapists and medical dosimetrists, particularly within teaching environments. Adams, et al.Reference Adams, Marks, Pawlicki, Hayman and Church 4 argue that the current scope of practice and clinical practice management for radiation therapists does not specifically include peer review nor does it explicitly use the term. The recent discussions on peer review of treatment plans have involved mainly radiation oncologists, yet radiation therapists and medical dosimetrists play a significant role in the development and QA of treatment plans. This has resulted in lack of evidence on whether ensuring that radiation therapy and medical dosimetry graduates from universities equipped with sound knowledge of peer review of treatment plans can improve the quality of treatment plans and minimise the threat to the overall quality and safety of radiotherapy care.
Moreover, there is lack of evidence on how students learn planning skills in university environments. At Queensland University of Technology (QUT), undergraduate radiation therapy students spend a significant amount of time in treatment planning courses using PinnacleReference Dutta, Balaji, Murli, Sudahar, Gopalakrishna and Potharaju 3 v14 (Philips Radiation Oncology Systems, Madison, WI, USA) and Monaco v 5.10 (Elekta CMS, Maryland Heights, MO, USA) systems. Therefore, implementing peer review of treatment plans might contribute to improved knowledge and skills on treatment plan production and critical evaluation. This narrative review summarises the clinical benefits, addresses the pedagogical benefits and the implementation of peer review principles in medical dosimetry and radiation therapy teaching environments focussing on the review of individual treatment plans.
Defining Peer Review
The term peer review has been defined in a variety of ways in the literature.Reference Adams, Marks, Pawlicki, Hayman and Church 4 , Reference Marks, Adams and Pawlicki 5 , Reference Brundage, Foxcroft, McGowan, Gutierrez, Sharpe and Warde 7 , Reference Chamunyonga and Bridge 8 Kaewlai and AbujudehReference Kaewlai and Abujudeh 11 defined peer review as ‘an evaluation by a colleague, who could be of the same or a different discipline working in a practice’. The word ‘peer’ refers to people in the same profession who are of the same or higher ranking.Reference Brundage, Foxcroft, McGowan, Gutierrez, Sharpe and Warde 7 Therefore, peer review of treatment plans represents evaluation of certain aspect(s) of a treatment plan that has been developed by another radiation oncology professional. Systematic peer review refers to a continuous, systematic and critical reflection and evaluation of performance using structured procedures.Reference Kaewlai and Abujudeh 11
Informal Versus Formal Approaches
Informal peer review methods in teaching often refer to situations where students solicit advice from other students.Reference Boehm and Bonnel 12 For instance, Boehm and BonnelReference Boehm and Bonnel 12 discussed how senior students from previous semesters’ courses who are sometimes viewed as subject experts may provide informal feedback to junior students. Similarly, in clinical practice an informal review process has the potential to facilitate reflective practice, improve staff motivation and help foster a culture of quality and safety in radiation oncology.Reference Chamunyonga and Bridge 8 When faced with complex treatment planning clinical situations, radiation therapists and medical dosimetrists often seek feedback and guidance from peers in the department. Senior radiation therapists routinely review treatment plans before they are clinically accepted. The process often involves engaging a colleague in dialogue in order to improve the quality of a developed treatment plan. This face-to-face discussion is a crucial part of plan evaluation as it can lead to changes in the departmental approaches to future treatment planning activities. This discussion requires social skills and experience in engaging in a dialogue without intimidating a peer. Ensuring that radiation therapy and medical dosimetry graduates have these capabilities is an added benefit to professional practice.
As a formal process, peer review can be used by radiation therapy and medical dosimetry students to evaluate their peers in a teaching and learning setting with the use of standardised peer review forms provided by the lecturers. The feedback gained can be used by students to critically re-evaluate their understanding of treatment planning concepts.
Lessons from Clinical Practice
Peer review of treatment plans, whether formal or informal, can directly impact the quality of care in radiation therapy.Reference Brunskill, Nguyen and Boldt 13 – Reference Moeller, Liang and Bobo 17 A recent study by Matuszak et al.Reference Matuszak, Hadley and Feng 9 present the benefits of a pre-treatment peer review process by an independent physician, physicist and dosimetrist, which resulted in changes in nearly one-quarter of SBRT patients, potentially preventing suboptimal treatments. In another study, Lo et al.Reference Lo, Liu and Chan 10 used the concept of peer review with a particular focus on structure contouring for SBRT plans. Their results showed significant changes in lung SBRT plans; the recontouring of several plans revealed violations of dose limits, most often involving inadequate planning target volume (PTV) coverage. Lo et al.Reference Lo, Liu and Chan 10 argue that peer review, especially of target volume delineation, is warranted to improve the consistency and quality in lung SBRT planning. In another study, Rouette et al.Reference Rouette, Gutierrez and O’Donnell 15 recommended that peer review be implemented before treatment to avoid replanning and adverse outcomes. In this study, changes in target volume, organs at risk (OAR) and dosimetric issues were prevalent and minor to major recommendations were reported in various tumour sites.
These studies show that peer review of target and normal structure contouring is not only feasible but is a necessary component of treatment planning. Lo et al.Reference Lo, Liu and Chan 10 emphasises that the identification of contouring variations with dosimetric impact on peer review supports the implementation of an expanded, therefore more rigorous peer review QA process for non-small cell lung cancer SBRT planning.Reference Lo, Liu and Chan 10 Drawing on the evidence from these studies, embedding a culture of peer review in educational institutions could be vital in consolidating students’ understanding of OAR contouring, treatment plan evaluation and its impact on treatment outcomes and patients’ quality of life.
Pedagogical Benefits
In addition to its potential in supporting understanding of treatment planning concepts, peer review in educational settings has several potential benefits to students. For instance, student engagement has been highlighted as one of several pedagogical benefits of peer review and peer-led learning activities in higher education.Reference Boehm and Bonnel 12 It is possible that it could help build students’ expectations and confidence in treatment planning activities. For instance, one peer is expected to take a direct pedagogical responsibility by creating learning opportunities through questioning, clarifying and scaffolding.Reference Chi, Siler, Jeong, Yamauchi and Hausmann 18 This form of peer-led learning is an effective means of encouraging student engagement and successful learning and can substantially enhance skills without diminishing content.Reference Drane, Smith, Light, Pinto and Swarat 19
Another desirable benefit is the notion that students can learn more from being producers of feedback than from receiving feedback reviews.Reference Gormally, Evans and Brickman 20 As a form of active learning, appropriately embedding peer review can provide personal contact and communication and persistence.Reference Braxton, Jones, Hirschy and Hartley 21 In active learning students ‘do things and think about the things that they are doing’.Reference Bonwell and Eison 22 Micari and DraneReference Micari and Drane 23 suggest that students in peer groups can share their own ideas and be practice giving explanations, hear other students’ ideas and be exposed to other students’ problem-solving processes. It is desirable that peer review be able to achieve these things for radiation therapy students.
With the shift from the traditional approaches to teaching and learning where the teacher is considered the expert,Reference Bloom, Dole and Kowalske 24 there is a need for educators to take into consideration the learner’s needs in designing pedagogies that engage the students in authentic tasks so that the skills can be easily transferred to practice. In treatment planning computer rooms, there is often limited opportunity for the students to develop higher level of critical thinking and an evaluative experience similar to that of clinical practitioners outside of the assignment tasks. Similar to the clinical contexts, peer review in learning contexts can benefit radiation therapy students by contributing to a collegial environment of transparency, and offering opportunities to develop social skills. Moreover, there is opportunity for students learn more as they think through, articulate and elaborate on their own arguments; hear and respond to ideas that challenge their own and observe others’ problem-solving practices in treatment planning.Reference Gormally, Evans and Brickman 20 This occurs when students discuss their treatment plans and compare their thinking with peers, requiring both regulation of their own cognition and influence on how peers learn.Reference Volet, Vauras and Salonen 25
Peer Review Implementation
Evidence-informed approach
One approach to the implementation of peer review in university settings is the use of an evidence-informed approach. As highlighted in Figure 1, it is possible to take into consideration approaches that have worked in clinical settings, learning from challenges and the barriers faced and using this evidence to support an effective approach in the academia.
An evidence-informed approach to the design and implementation peer review activities in undergraduate radiation therapy training.
Analysis of cohort needs and approaches to teaching
Embedding peer review requires that academics take into consideration the principles of effective curriculum design. For instance, the use of the modified Bloom’s taxonomy,Reference Athanassiou, McNett and Harvey 26 which provides a classification of the levels of thinking during the learning process, can be used to ensure that the peer review tasks are embedded at an appropriate level of complexity in learning, as well as treatment planning. If the peer review is to be embedded in a final year course; based on the Blooms’ taxonomy,Reference Athanassiou, McNett and Harvey 26 students are expected to critically evaluate their practice. Therefore, critical analysis of SBRT treatment plans with intensity modulated radiation therapy or volumetric modulated arc therapy treatment planning techniques is essential in the final year courses, whereas peer review of three-dimensional conformal radiation therapy treatment planning tasks can be embedded in low-mid level of learning as the emphasis on critical evaluation is less.
Clear rationale and scope of review
Designing a peer review activity requires a clear rationale and objectives that are informed by a contextual analysis. Activities can be designed to address a specific problem or challenge in treatment planning. For instance, in a cohort with students from diverse backgrounds, there may be a need to improve student engagement. In such cases the teachers can facilitate the pairing process so that there is dialogue between students with different skill. Peer review activities can also target a particular skill set. This requires that academics identify an area that will benefit the students most. For example, peer review tasks could be designed to focus on one or several of the following areas:
-
∙ OAR delineation.
-
∙ Isocentre and reference point placement.
-
∙ Beam arrangement and monitor unit weighting.
-
∙ Determination of whether the plan meets OAR tolerance and PTV coverage objectives.
-
∙ Overall plan review and evaluation of acceptability for clinical use.
Process and documentation
Peer review of treatment plans can involve students evaluating each other’s treatment plans, documenting the outcomes and providing constructive feedback. Commenting on each other’s work is a one way to promote the development of such critical evaluation skills. A paper-based approach can be used to provide feedback to another student by documenting the changes required. For instance, in the study by Lo et al.,Reference Lo, Liu and Chan 10 for each structure that was contoured, the reviewers examined the original contours and assigned one of the following scores: ‘major change required’, defined as original contour unacceptable; ‘minor change required’, defined as original contour still acceptable; ‘no change required’; or ‘missing contour’. If the scoring was discrepant between two reviewers (one selected major change, whereas the other selected minor change or no change), the case was distributed to a third oncologist for additional peer review.Reference Lo, Liu and Chan 10 Likewise, any discrepancy between the two students can be reviewed by the academic staff members.
As indicated in Figure 2, a systematic peer review workflow that can be implemented in the learning and teaching context. In the practical sessions, students can be provided with a task sheet that has clear instructions on how the peer review process is to be conducted against set criteria. The task sheet can also be used to provide instructions on the pairing process and allocated time for peer review. A standardised peer review form developed by academics can then be used as a tool to evaluate certain parts of a treatment plan produced by a peer. This paper-based evaluation can be followed by a brief face-to-face discussion.
Systematic peer review process for individual treatment plans in undergraduate learning.
Potential Challenges
Positive attitude and methodological considerations
Promotion of a positive attitude towards peer review, establishment of well-thought-out methodology, stimulation of change in the students’ performance are all important aspects of setting up a peer-reviewed process.Reference Kaewlai and Abujudeh 11 It is also important for the academics to recognise that peer review may have unintended negative effects for some students, and addressing these is an important part of ensuring that it meets its goals. Therefore, setting up peer review would require good preparation and management.Reference Kaewlai and Abujudeh 11 Education and training in specific peer review tools is needed for the students to be prepared for peer review in practical sessions. Moreover, information on the process and expectations and approaches to peer review as well as the potential challenges students can face in the peer review sessions is essential to the success of each review. Finally, the structure and process for providing and gaining feedback from peer review need to be clearly identified. This information can be provided to students in a lecture format as a value proposition for them to develop interest in the review. Didactic presentation, peer review demonstration, role play involving sample clinical situations highlighting peer review principles, and discussion of challenges in implementing peer review illustrated ways in which key peer review principles could be applied.Reference Semper, Halvorson, Hersh, Torres and Lillington 27
Constructive feedback and culture
One of the important skills required for effective peer review is the ability to communicate and give and receive constructive feedback.Reference Semper, Halvorson, Hersh, Torres and Lillington 27 The need to build listening skills and having the ability to give and receive constructive feedback were consistently identified as important.Reference Semper, Halvorson, Hersh, Torres and Lillington 27 A feedback loop, whether to an individual, group or organisation, is a critical part of peer review.Reference Kaewlai and Abujudeh 11 George and HaagReference George and Haag-Heittman 28 suggest that fear of retaliation may tarnish the notion of peer review and could be a barrier to honest feedback. In addition, they highlighted the lack of constructive feedback and inflated affirming feedback as contributors to the negative connotation that professionals may have about peer review processes. Therefore, for peer review to be successful an open and honest partnership must be established between peers. It is also important that honest partnerships are formed from individuals who are unlikely to be in competition with each other for promotion opportunities and are willing to engage in open discussion without fear of offending a peer. Other researchers have referred to the ‘halo effect bias’ arising from a reviewer who has positive feelings for their reviewee and thus provides more favourable feedback than their performance would merit.Reference La Lopa 29 Moreover, the success of peer review has also been attributed to a positive peer-reviewed culture and a commitment to a team.Reference Kaewlai and Abujudeh 11
Creating a true culture of peer review will require leadership involvement and long-term strategies to stimulate and foster this change in teaching radiation therapy planning principles. In spite of these challenges highlighted, there is no doubt that the value of peer review outweighs logistical issues.
Conclusion
Providing safe and high-quality treatment is imperative in radiation therapy; however, there is a need to apply effective teaching methods and appropriate use of technology at undergraduate level, driven by the current clinical needs. Evidence from clinical studies suggests that peer review of treatment plans can improve planning consistency, decrease the need for replanning and improve quality of care through the safe delivery of high-quality radiotherapy plans. Peer review as a pedagogical tool has the potential to improve students’ treatment planning skills through peer-led learning. The feedback process improves student engagement and communication, with students being providers as well as recipients of feedback on their treatment plans. Radiation therapy and medical dosimetry students can also benefit with the development of critical appraisal skills and reflective practice. Research is required to evaluate the possible impact on student learning and future clinical practice.
Acknowledgements
The authors acknowledge the colleagues at the QUT for their support.
Financial support
This review article received no grant from any funding agency, commercial or not-for-profit sectors.
Conflicts of Interest
None.
