Research overview

Starting in February 2020, we conducted an assessment to inform our development of an EDC for the Transitions Clinic Network: Post Incarceration Addiction Treatment, Healthcare, and Social Support (TCN PATHS) study. TCN PATHS is a NIH-funded hybrid type-1 effectiveness/implementation RCT assessing the effect of the Transitions Clinic Network’s (TCN) model of primary care on engagement in OUD treatment among individuals with OUD recently released from incarceration [Reference Howell, Puglisi and Clark11]. The core of the TCN model is the inclusion of community health workers into the primary healthcare team. TCN’s community health workers have a history of incarceration and have an active role in the patients care team, as they provide social support and address social determinants of health.

We are recruiting participants from five sites across the continental United States and Puerto Rico. Once enrolled, a participant is randomized to receive primary care from either a TCN program or a standard primary care program. From a translational lens, TCN PATHS focuses on clinical research in evaluating the effectiveness of the outpatient TCN primary care model and concurrently evaluates the implementation of the model.

Conceptual framework: knowledge-to-action

In developing CRICIT, we used the Knowledge-to-Action (KTA) conceptual framework, which is used to facilitate the knowledge translation process in health care [Reference Graham, Logan and Harrison12,Reference Straus, Tetroe and Graham13]. KTA has two parts; at the core is the knowledge creation funnel and encompassing the funnel is the action cycle, as depicted in Fig. 1. Within the knowledge creation funnel, knowledge gained from primary studies and synthesizing knowledge on a particular topic are developed into guidelines or tools to assist in knowledge dissemination. As activities are conducted during this phase, the knowledge becomes more digestible and approachable to the end user and stakeholders. Once knowledge is transformed, the next step is to enter the action cycle by identifying, reviewing, and selecting the knowledge that is required to be implemented. From here, there are six additional steps in the action cycle: adapt to local context, assess barriers and facilitators, tailor interventions, monitor use, evaluate outcomes, and sustain use. Due to the complex nature of knowledge translation, these actions may be implemented in a nonlinear manner.

Figure 1.

Knowledge-to-action framework. Graphical representation of the knowledge-to-action framework created and used with permission from “knowledge translation in health care: moving from evidence to practice” [Reference Straus, Tetroe and Graham13].

Although the KTA framework was developed to move knowledge into healthcare practice, we found KTA to be a helpful framework to guide the development and implementation of our EDC system. This framework was selected as common components of the software development life cycle (SDLC) can be mapped to the KTA domains, creating an interdisciplinary framework that leverages knowledge translation principles in the context of the SDLC [Reference Malik and Nigam14]. Table 1 contains a mapping between common SDLC steps and the KTA domains, which demonstrates the alignment between the two.

Table 1.

Framework mapping alignment between the common SDLC steps and the Knowledge-to-Action framework

We created knowledge based on the original grant application and synthesized this into an implementation protocol to document what activities were required to accomplish the primary aims of the study. We facilitated conversations with study stakeholders to determine the gap between the required activities and the necessary actions to complete them. Working closely with stakeholders, we iteratively moved through both the knowledge generation funnel and the action cycle as we developed and implemented our EDC to include components of the original protocol and significant protocol adjustments that were made for numerous reasons, including adaptations due to COVID-19. During the design phase, we identified which aspects of the protocol needed to be adapted to local context, for example, access to site-specific consent forms and Spanish versions for our Puerto Rico site. The design phase also included identifying barriers and facilitators to protocol implementation and which design elements could be used to overcome barriers. During the implementing and testing phase, we programmed and tested various design elements and implemented the elements that provided the best solution to overcome barriers. Once testing was complete, we deployed our software then monitored use and evaluated outcomes by reviewing data, observing users in the system, and conducting group conversations to evaluate programming approaches. The maintenance and support phase included implementing new features and innovations to adapt to protocol changes and user needs.

TCN PATHS EDC development

Using the implementation protocol that was developed during the knowledge creation phase, we identified stakeholders to be included in the EDC development process. Table 2 highlights the different roles of our stakeholders, how they interact with the EDC, and the key perspectives they contribute to the development process.

Table 2.

Stakeholders’ key perspectives stakeholder’s interaction with electronic data capture systems (EDCs) and key perspectives

Within the stakeholder roles, our research team spans multiple disciplines, which highlights different approaches and needs when developing the EDC. During the EDC development, we integrated perspectives and insights from multiple disciplines and fields of study including: bioethics, clinical research, computer science, data science, education, instructional and graphic design, medicine, public health, survey research, and statistics. Coinvestigators included statisticians, implementation scientists, and physicians who would participate in study data analysis. The data manager/project manager (KC) provided interdisciplinary leadership, leveraged the perspectives of the different roles and disciplines, and helped to establish common ground and identify specific components to meet the needs of the various EDC users. University software engineers and database administrators provided technical assistance regarding security and best practices in database design.

Through continuous conversations with stakeholders, we identified several foundational goals of our EDC. The system would need to support a complicated clinical research protocol. We use the term complicated, as the activities within a protocol are easily understood and are predictable (e.g., screening will result in a person being eligible or not), as opposed to the unpredictable and nonlinear nature of complex systems [Reference Sturmberg, Martin and Katerndahl15]. Although each component of the protocol is relatively simple, the synergy of multiple tasks and conditional procedures leads to a complicated protocol. System users would use the EDC to input and store various data about eligibility screening, participants enrolled, and activities to be completed, and to run summary reports that turn this data into information. Project management and coinvestigators would require a centralized system to oversee both site-specific and project-wide data. To the research staff, a centralized system meant that core study components (e.g., screening, randomization, enrollment, etc.) were contained within one system and did not require multiple software. Quality controls and data checks were essential to all stakeholders to ensure data integrity and adherence to the implementation protocol. Given that TCN PATHS is a longitudinal study with 13 study visits over the course of 12 months, stakeholders agreed that a robust participant tracking system would be an essential feature.

CRICIT architecture

CRICIT was built using the Claris FileMaker Pro (FMP) software [16]. This software supports the rapid development of relational databases. Yale University has a FMP license and the Information Technology (IT) Department provides FMP server and administrative support. CRICIT is located on a HIPAA and electronic protected health information-compliant server that is also used by Yale New Haven Hospital’s Emergency Department. This server is backed up every hour. Prior to release, the database was reviewed by Yale University’s IT staff to ensure the system met security standards. CRICIT is enabled with encryption at rest to protect the system from a data breach and encryption in transit for network connections to the server. CRICIT was developed in the context of good clinical practice and complies with 21 CFR Part 11 including: limit system access and authority checks, utilization of an audit log, and system checks to assure appropriate sequencing of research activities [17]. Access to CRICIT is controlled by individual user accounts, passwords, and privilege sets. Research staff have a privilege set as “Data Entry,” which limits their capabilities to access and edit some data in contrast to the “Admin” privilege, which has full access to all data and editing privileges. The audit log documents the date and time at which changes were made to the data and the user responsible for the change.

CRICIT was programmed with system check techniques to assure appropriate sequencing of research activities. CRICIT uses data specific fields to prevent data errors and limit the use of open text fields. There are fields specific for dates, numbers, and text string dropdown menus, which will only allow users to input specific values. The data stored in these fields are used to trigger code that will make the next step in the research protocol available. This type of programming logic is used extensively in CRICIT to maintain fidelity to the protocol and assure correct sequence of activities.

CRICIT is a relational database, which allows data to be stored in separate tables, but because the data is related, a user is able to access data across tables. This is beneficial to our stakeholders in various ways. Our research staff is able to input and access data across tables for participant information (e.g., who is due for an appointment and if they are incarcerated) and our analytics team is able to access data within tables to conduct analysis (e.g., across the project how many participants completed an initial follow-up appointment). This allows a user to view both individual and aggregate data across various aspects of the protocol, as well as system metadata. The relational database design provides the structure and capabilities to support longitudinal studies. Fig. 2 is a graphical representation of the some of the key elements of CRICIT’s architecture.

Figure 2.

Database architecture. The rectangles represent data tables and a participant identification number links data across the tables.

CRICIT implementation

The concept of CRICIT was developed in March of 2021 and the first version ready for users was deployed in May 2021. The CRICIT development team was led by the project manager, KC, and received extensive coding and graphic design support from one of the research staff, MT. Authors CR and KRS provided software development coaching and mentorship. The initial deployment included core protocol components, such as the ability to screen individuals, conduct the consent process, enroll participants, complete locator forms, track participant encounters, and schedule study visits.

In consultation with our stakeholders, we have adapted our protocol multiple times to adjust how the research environment has been affected by COVID-19. For example, initially the inclusion criteria stipulated only enrolling participants when they were still incarcerated; however, restrictions to research staff accessing these settings required us to expand our inclusion criteria to allow enrollment of individuals who were not currently incarcerated, but were released from incarceration in the past 30 days. Due to the rapid development environment of our software, we were able to adapt CRICIT to meet the protocol changes efficiently and with minimal lag between protocol decisions and software implementation.

We used the KTA framework to guide the initial CRICIT deployment, as well as managing the implementation of ongoing adaptations as we identified opportunities to streamline workflows in an applied research setting. Throughout the KTA cycle, we utilized user-centered design techniques, such as observing research staff using CRICIT and continuously seeking feedback, to inform software development [Reference Holtzblatt and Beyer18,Reference Schleyer, Thyvalikakath and Hong19]. For each protocol component that was incorporated into CRICIT, we began by assessing any adaptations that were necessary for each site; for example, some sites were collecting data for supplemental studies, while others were not.

We assessed barriers and facilitators to implementing the protocol, such as the case of identifying anchor dates for surveys. Our protocol required us to identify survey timeframes that begin 90 days prior or the date of the participant’s most recent study visit. Survey data are collected in the web-based software Qualtrics, and key data are passed from CRICIT into Qualtrics [20] to customize the questions for participants (e.g., “Since July 12th have you…”). Identifying the correct anchor date for a survey was a significant burden on the research staff and created a barrier to their efforts to comply with the protocol. We identified multiple coding interventions to address this barrier, and selected the intervention that would result in the least staff burden and highest data quality. We wrote an algorithm that would retrieve study visit dates, calculate the anchor date for the selected survey, then, using query strings, pass the information into Qualtrics which populated the accurate timeframes for a given survey. This solution improved adherence to the protocol, reduced research staff burden, and reduced data entry error.

As we implemented new components and features, we monitored and evaluated the backend data collection and user experience, making adjustments as needed. One of the ways we sustained research staff’s knowledge of the protocol was through the use of informational buttons. As shown in the urine drug testing (UDT) example in Fig. 3, these buttons allow research staff to access protocol details and definitions, thereby reinforcing information from the protocol. Additionally, CRICIT provides real-time status information on protocol adherence including screening, enrollment, and completed visit statistics.

Figure 3.

Informational buttons. Screenshot of informational buttons for urine drug testing (UDT).

We utilized multiple approaches to manage changes during our rapid development cycle to protect the integrity of our system and data. Two databases were created, a development database and a production database. The development database contains synthetic data for purposes of testing and development. The development database is where we programmed and tested new features to assure they were working as intended. Our software has a debugging function that is used to review code line-by-line to assure functionality and identify any errors. Once programming was thoroughly tested, we migrated the structural changes to the production database. Structural changes do not affect the data in the production database. Research staff were alerted to system updates and provided instructions on how to report features that were not working as intended. The project manager maintained a revision log that tracked reported problems, new feature requests, and how/when these issues were resolved. Additionally, the university IT Department backs up our system hourly, daily, and weekly, which would allow us to revert to a previous version, if needed.