Program design and educational structure

The overall structure of the training program is depicted in Figure 1. The level definitions, contents, primary participation criteria, and implementation years of the program are presented in Table 1. Further details are provided in Appendix 1. The levels were established with reference to the framework defined by ACRP [11]. Participants were permitted to attend the program multiple times (repeating the same level or advancing to different levels) to encourage progressive skill development.

Figure 1.

The overall structure of the competency-based, level-specific training program.

Note: QMS = quality management system; RCA = root cause analysis; CAPA =corrective and preventive action; RBA = risk-based approach; SDV = source data verification; SDR = source data review.

Table 1.

Level-specific structure of the training program

Table 1. Long description

A table comparing training program levels, target profiles, eligibility, and workshop contents. The table has four rows and three columns. Column headers are Level and implementation year, Target profile and main eligibility, and Contents of the workshop. Row 1: Level and implementation year, Entry (2022); Target profile and main eligibility, Profile: Understands instructions and reliably carries them out. Eligibility: No prior experience in monitoring.; Contents of the workshop, Acquire basic knowledge (regulations, monitor roles, medical records etc.), Experience site monitoring on an introductory basis. Row 2: Level and implementation year, Foundational (2020-2024); Target profile and main eligibility, Profile: Developing independent thinking skills. Eligibility: <2 years of monitoring experience (or planned future engagement).; Contents of the workshop, Protocol review for potential risk identification, Role-playing site monitoring (SDV/SDR, CRC/ investigator inquiries, monitoring report drafting), Ethics committee approval/notification document review. Row 3: Level and implementation year, Early-Intermediate (2020-2024); Target profile and main eligibility, Profile: Understands study objectives; capable of independent thought and action. Eligibility: 2-5 years of monitoring experience; experience/plans in developing monitoring plans.; Contents of the workshop, Study risk identification and risk assessment, Practical development of risk-based monitoring plans. Row 4: Level and implementation year, Intermediate (2023-2024); Target profile and main eligibility, Profile: Understands long-term study goals; capable of recommending/supporting revisions of the research plan. Eligibility: ≥ 5 years of monitoring experience; lead monitor for ≥ 3 protocols with ≥ 1 full study cycle completed.; Contents of the workshop, Discussion of CAPA with CRCs, Monitoring team leadership and coaching.

Note: SDV = source data verification; SDR = source data review; CAPA = corrective and preventive action; CRC = clinical research coordinator.

Each level comprises a pre-assigned learning task, followed by a one- or two-day workshop. Pre-assigned learning tasks, tailored to each level, include activities such as watching webinars to acquire foundational knowledge and reviewing materials related to the upcoming workshops. This approach not only helps participants transition smoothly into the workshop but also enables them to gain knowledge that cannot be fully covered during the workshop alone, allowing for a more efficient use of the limited workshop time.

The workshops across all levels comprise a lecture followed by practical, interactive group work. In the program design, emphasis was placed on three key elements: “practice-oriented learning,” a “supportive learning environment provided by facilitators,” and “mutual interaction and learning among participants,” to ensure the effective acquisition of competencies. Practice-oriented learning is intended to translate knowledge into real-world application. A supportive learning environment aims to ensure psychological safety and encourage active participation. Mutual interaction and learning are designed to serve two purposes: fostering multifaceted perspectives through discussions among participants and facilitators with diverse backgrounds and experiences, and expanding professional networks among ARO monitors. For example, at the Foundational level, participants attend a lecture on site monitoring to strengthen their understanding, followed by a role-play exercise in which they simulate the monitoring process under facilitator guidance. Specifically, during group work, participants engage in the entire sequence of the monitoring process, including preparation before monitoring, reviewing source documents, querying investigators, and drafting a monitoring report. In designing the group work, participants were intentionally assigned to groups to ensure diversity in professional roles and institutional affiliations.

Operational framework of the program and continuous improvement

The program was managed by an operational team composed of ARO monitors, all of whom had at least two years of practical experience. Team members developed a detailed manual and also served as group work facilitators based on it.

After each program session, the team conducted a debrief using the KPT (Keep-Problem-Try) framework – a structured reflective methodology. Insights gained through this process were analyzed together with participant needs and feedback, in light of evolving domestic and international clinical trial environments. These comprehensive evaluations were reflected in annual adjustments to training levels, as well as revisions to the program content, training materials, and facilitation manual. This approach provided an operational framework for continuous improvement in educational quality.

Implementation of surveys

Two types of surveys were conducted among participants to evaluate the program. The first type, “post-workshop survey,” was administered to participants across all levels between 2020 and 2024, within a week after completion. The questionnaire included both closed- and open-ended questions assessing various aspects such as participant backgrounds (affiliation and primary professional role), satisfaction scores related to the workshop (pre-assigned learning tasks, lectures, and group work), and overall impressions. While the number of satisfaction questions was optimized each year according to the program’s content, a consistent 5-point Likert scale (1 = “very dissatisfied,” 5 = “very satisfied”) was employed.

The second type, “follow-up survey,” aimed to assess the mid-term outcomes that were difficult to evaluate immediately after program participation. This survey was conducted in 2022 and 2024, targeting individuals from all levels who participated in the program between 2020 and 2024, within two years following their participation. This survey included “post-program engagement in monitoring activity,” “post-program utilization of training materials,” “utility of learning from training program in daily practice,” and “post-program participant networking.” For both surveys, participants were contacted via email and asked to complete an online questionnaire (Google Form). To illustrate the specific structure of each survey, the latest 2024 versions of the questionnaires are provided as representative examples in Appendix 2.

Quantitative analysis of questionnaires

Descriptive statistics were used to evaluate the distribution of respondent backgrounds and assessment scores. In the post-workshop surveys, multiple responses were allowed for the question regarding primary role; therefore, the total number of responses exceeded the number of respondents. Median and range of satisfaction scores were calculated for each year. Although the number of satisfaction-related questions varied slightly by year, all available data were included in the analysis. Regarding pre-assigned learning tasks, although both mandatory and optional tasks were provided, only the former were included in the evaluation.

In the follow-up survey, the next key metrics were analyzed. (1) “Post-program engagement in monitoring activity” defined as having conducted at least one monitoring activity after participating in the program; (2) “Post-program utilization of training materials” calculated as the percentage of respondents who reported specific use (using materials for monitoring activities, institutional training, or self-study); (3) “Utility of learning from training program in daily practice” calculated as the percentage of respondents who selected 4 or 5 on a 5-point scale; and (4) “Post-program participant networking” calculated as the percentage of respondents who selected any of the items indicating increased interaction or networking with other participants or facilitators.

Qualitative analysis of questionnaire

To analyze the factors contributing to participants’ evaluations, the corresponding author applied an a priori framework based on the three key elements emphasized by the operational team during program design – practice-oriented learning, a supportive learning environment provided by facilitators, and peer interaction and mutual learning. Using this framework, a descriptive coding analysis was conducted via a deductive approach on the free-text responses from the post-workshop surveys (2020–2024). These comments were independently categorized by three coders (Y.K., S.I, and M.K.), with any discrepancies resolved through discussion to reach a consensus. Comments corresponding to multiple categories were assigned to each applicable category, and the proportion of comments within each category was calculated.

Ethical considerations

The ethical review was waived because the educational program was not subject to Japan’s Ethical Guidelines for Life Sciences and Medical Research Involving Human Subjects. All participants agreed to complete the self-assessment questionnaires used to examine their understanding of the program content. The survey forms included a statement explaining that responses in a de-identified format might be disclosed externally and that submission of the form implied consent to this condition.