Discussion

The principles of QbD have been developed and adopted in various industries, including in pharmaceutical manufacturing. More recently, their potential application in clinical research has been described through multistakeholder efforts, which have included regulators, members of academia, as well as the pharmaceutical and medical device industry. While there are many resources available that describe QbD concepts as they relate to biomedical research, specific examples for adoption and implementation of this approach have remained lacking. In this manuscript, we have described a formal pathway for implementation of QbD at an academic health center using identification and prioritization of CTQ factors, which are key components of this approach. We achieved this through establishment of a formal CTQ-DS with the goal of embedding QbD principles into the design and management plan of clinical research protocols proposed at our institution. During the past 2 years, we conducted eight CTQ-DS, each reviewing a unique clinical research study. The majority of the meetings were held virtually due to COVID-19 pandemic restrictions.

We continuously improved the planning, conduct, and reporting process for our design studios and based on the latter knowledge, we devised a standard operating procedure that can be adapted for settings outside of our institution. Finally, we conducted a survey to evaluate the participating PIs’ perception of value and the utility of CTQ-DS in improving the design and conduct of their study. The results show that CTQ-DS were well received and have played a valuable role in helping PIs proactively anticipate aspects of the trial protocol that were deemed to be important and adjust their protocol accordingly to address these findings. The only study that did not report protocol changes in response to the CTQ-DS was a multisite clinical trial. This finding is not surprising given the many difficulties involved in changing the design of studies that have already been finalized, especially multicenter studies, which have already gone through many iterations and are less amenable to change. Therefore, our findings indicate that CTQ-DS are most useful for clinical studies at the design stage. However, it is possible that even for a finalized protocol the findings from a CTQ-DS could still be helpful in creating a site-specific trial management plan and improving the feasibility of the study at a single implementing institution, although this remains to be determined in future follow-up evaluations.

There are several important aspects of our efforts that merit highlighting. In recent years, there has been increasing recognition of the critical need for a team-science approach to the development and conduct of translational research [Reference Selker27–Reference LeClair, Kotzias and Garlick30]. This approach promotes engagement of a multidisciplinary, multistakeholder team in the design, conduct, and dissemination of clinical research in order to generate knowledge that is impactful and of relevance not only to the scientific community (researchers), but also to the society as a whole. Our QbD implementation methodology has been uniquely suited to facilitate team science and stakeholder engagement. In our CTQ-DS, we included members with a wide range of expertise and skillsets to help identify the diverse CTQs unique to each study. The team science approach was manifested not only by including multidisciplinary members in our core team but also through involvement of ad hoc content experts, community representatives, and members. Additionally, the formative feedback from CTQ-DS which was communicated in a collegial and helpful manner created an environment of collaboration rather than evaluation. Therefore, the design studios were offered as a resource to help better design and manage clinical studies by the research teams rather than a regulatory hurdle. In this feasibility study, PIs had the option to include members of their study team in the studio, but few availed themselves of this opportunity. A case could be made that including study team members would increase the likelihood that CTQs would be successfully addressed in the protocol. Future evaluation of the studios should incorporate this variable in the evaluation design

The role played by participant advocates in the design of clinical trials is also critical. Novel approaches, such as the use of community engagement studios [Reference Joosten, Israel and Williams31], are increasingly embraced by CTSA hubs in an effort to embed the unique voice of patients and prospective volunteers in early phases of trial design. Clinical Trial Transformation Initiative has also emphasized the beneficial role played by patient and community advocates in QbD [Reference Patrick-Lake32]. We found that inclusion of patient/participant advocates in our design studios was an effective method for incorporating the perspective of study subjects into the design and implementation plan of clinical studies. In addition, their input played a key role in several aspects of study design, most notably in the recruitment and retention plan as well as selection of patient-relevant endpoints.

The implementation of QbD via CTQ-DS not only was well received by our PIs as a resource to help them better design and develop their studies but it also provided an opportunity for our researchers to incorporate the latest recommendations of the upcoming ICH E8 and ICH E6 renovations into their clinical research protocols [33,34]. These internationally recognized guidance documents, which cover good clinical practices for clinical trials, now include a section on the consideration of quality in the design and conduct of clinical studies. Furthermore, special emphasis has been placed on identification and management of CTQ factors. Therefore, adoption of QbD principles through implementation of CTQ-DS has helped our team stay at the forefront of these efforts aimed at incorporating quality into clinical research design and development. Quality in this context is defined by Clinical Trial Transformation Initiative as “lack of errors that matter,” such as errors that compromise patient/participant safety or the integrity of the data and results of the study. Future work extending the current findings will evaluate whether the CTQ-DS is effective for preventing errors that matter in the conduct of clinical research being carried out at an academic health center.

Some of the limitations of our approach also need to be mentioned. Our system for applying QbD concepts is especially suited for academic institutions and adoption of a similar scheme in smaller organizations may be challenging. At academic health centers, we have access to a wide pool of experts to populate our working group. Moreover, these individuals were willing to participate as an element of their University Service and did not require compensation. This approach may not be feasible in smaller organizations or those with differing priorities. However, we believe that the concepts highlighted in our study (such as team science, community engagement, and design studios) can be adapted and tailored for other settings, including commercial settings. For instance, partnerships can be formed between multiple organizations with the goal of creating CTQ-DS that can help review a variety of external and internal studies, much like central and commercial institutional review boards. Another important limitation of the QbD approach is that it remains to be demonstrated that adoption of QbD or any other mechanism for incorporating quality metrics into clinical research will in fact improve quality of clinical studies. Therefore, it is important that future, carefully designed and controlled studies address this critical question (whether adoption of QbD or other quality assurance methodologies in fact improves quality of clinical research). In addition, relevant qualitative and quantitative endpoints need to be identified and validated in order to help with evaluation of QbD efficacy in improving quality of clinical research. In fact, we are planning studies to address these important gaps in order to plan future larger studies examining QbD effectiveness. Nevertheless, a formalized approach for implementation of QbD at academic health centers, which can then be adopted at other institutions, will be the critical first step to enable future research to test the comparative effectiveness of QbD (as opposed to current common practices) in improving quality in clinical research. Hence, our approach for operationalizing QbD can serve as a platform for adoption of QbD at other academic health centers and thereby facilitate future multicenter effectiveness evaluation of the QbD approach. Finally, there are pitfalls that we encountered during the implementation of our approach that other investigators need to be aware of and account for in their planning. First, CTQ-DS require participation of many faculty, staff, and other participants and this requires time commitment and coordination of schedules for a sizeable number of individuals. Sessions need to be planned and scheduled 1–2 months in advance in order to facilitate this process. Second, identification of clinical studies and PIs who are eligible and willing to participate in design studios requires planning and support from organizational areas involved in clinical research (Center for Clinical Research, Institutional Review Boards, etc). We found that this process needs to be planned several months in advance in order to help successful identification of a project and to assist PIs with some of the requirements of this process (i.e. Study synopsis generation, creation of a study presentation, education regarding the QbD process and design studios). Preplanning is also critical to include patient representatives in the design studios. Most of our patient representatives were identified and invited by the PI of the under review, but in cases where these PIs lacked such contacts, it was useful to reach out to the Community Engagement Unit of our Clinical and Translational Science Award hub for assistance. Lastly, ongoing surveys and feedback from participants have been used to optimize many of the operational aspects of the design studios. These included the CTQ identification a scheme and the scoring system used to prioritize CTQs. Therefore, continuous evaluation and improvement of the CTQ-DS processes are crucial to the successful implementation of this approach in a changing academic environment.

In conclusion, we have demonstrated that adoption of QbD concepts is feasible through administration of multidisciplinary design studios focused on CTQ factors specific to each study. Moreover, the studios were perceived as extremely valuable by the participating PIs and stimulated PIs to address the identified factors in their protocol revisions. Future studies will need to determine whether implementation of QbD using this approach improves quality of clinical research at academic health centers.