Team Science Leadership Curriculum

The Team Science Leadership curriculum is outlined in Table 1 and discussed in more detail in Supplementary Materials.

Table 1.

Elements of the Team Science Leadership curriculum

We introduced the Team Science Leadership initiative to the Clinical Scholars by providing them with a Team Science Leadership Self-Assessment document with a series of questions about

Team Science Leadership for self-assessment (Table 2), and if they chose, discussion with their primary mentor and/or the leaders of the Clinical Scholars program. The document covers the Scholar’s experiences and/or views of current and past experience in participating in teams, developing and articulating a vision for a team, delegation of tasks, oversight of team members, decision-making, conflict resolution, models of leadership and role modeling, and self-reflection on the Scholars’ current strengths and weaknesses as a team leader. In addition, Scholars rate themselves on their Team Science Leadership Competencies (see below) on entry into the program and yearly using the same survey used by the TSE/Es. To broaden the Scholars’ scope in contemplating their roles as leaders, we searched for other leadership models that might be instructive for comparison. We selected a program on leadership, The Art of Teams: Achieving Excellence as Equals, that uses the relationship among the members of a string quartet as a paradigm of shared and shifting team leadership, reflecting the relative roles of the instruments as dictated by the music and the overall team structure of the quartet [Reference Wolff7].

Table 2.

Rockefeller University Clinical Scholars Team Science Leadership self-assessment survey

Team Science Leadership Competencies and Teams Science Leadership Survey

As TSE/Es, we collectively recognized the need to define the competencies for effective Team Science Leadership for translational research studies. By combining our domain-specific expertise and experience, we created a list of competencies that we assessed to be essential for an outstanding translational scientist to lead a multidisciplinary team in conducting a human participant research protocol (Supplemental Figure 1). All 15 Senior Staff members participated in a modified Delphi method [Reference Schlesinger, Romanick and Tobin1,Reference Schneider, Evaniew, McKay and Ghert8,Reference Dalkey9] for generating and reviewing both the major domains and the items within domains. Over the course of 6 months of weekly meetings, the domains were first agreed upon, and then the items within domain were created, deleted, and revised until neither new items nor major revisions were suggested. These draft items were then shared with all of the current Scholars for feedback and revision. They were also shared with colleagues from Yale University and the University of Pennsylvania in a series of face-to-face meetings at each site. The list of competencies was finalized to begin the project, but with agreement that the competencies would need to be reviewed and modified periodically.

The final list converged on five domains: 1. Foundational Leadership Competencies containing nine items centered on creating and leading teams, creating and communicating a vision, building trust among team members, mentoring team members, and establishing lines of open communication; 2. Professionalism Competencies containing five items that describe aspects of responsibility and modeling ethical and supportive behavior; 3. Team Building and Team Sustainability containing 12 items that describe more granular level aspects of team communications, recognition of team member achievements, conflict resolution, seeking advice, and taking responsibility; 4. Appropriate Use of Resources and Execution of Study containing six items that describe elements of appropriate budgeting, staffing, overcoming obstacles, organizing continual quality improvement, and delegating responsibility appropriately; 5. Regulatory Accountability containing five items describing issues relating to compliance with Good Clinical Practice (GCP), federal and state laws and regulations, and issues related to data management, security, and sharing.

Prior to the Team Science Leadership initiative, we periodically reviewed our perceptions of the progress of individual Scholars during our weekly TSE/Es meetings. During these discussions, we appreciated the value of sharing our perceptions since we each observed the Scholars in different roles, interacting with different people, and performing different tasks. We thus recognized that by summing our individual observations we had a much more comprehensive assessment of the Scholars’ progress. We therefore set a goal of systemizing the information from all TSE/Es and combining it with the Scholars’ mentors’ evaluations and integrating it into a comprehensive plan for competency evaluation, feedback, and improvement. To achieve this goal, we developed a research protocol describing the evaluation of Scholars’ Team Science Leadership competencies and received approval from the Rockefeller Institutional Review Board prior to implementing the study. To evaluate the Team Science Leadership competencies of the Scholars, we converted the competencies into a survey in which each competency is rated based on a six-point Likert scale score (0–5), with additional space for free text comments (Table 3). We then implemented the survey in a REDCap [Reference Harris, Taylor, Thielke, Payne, Gonzalez and Conde10] format to facilitate data collection and preserve the confidentiality of the reviews. Each Scholar evaluates herself or himself, each mentor evaluates her or his Scholar, and each TSE/E evaluates each Scholar across as many items and dimensions as are appropriate, based on content area and exposure to the Scholar’s activities in that area (Supplemental Figure 2) The evaluations by the TSE/Es, mentor, and Scholars are combined into a written report that includes 1. the aggregated evaluations by the mentor and TSE/Es, presented as the mean ± SD score for each competency, 2. an adjacent column with the Scholars’ self-evaluation scores, 3. an adjacent column listing the difference in the aggregated score and the Scholar’s score for each competency, and 4. free text, anonymized comments by the TSE/Es and mentor. After field testing the survey with one Scholar, we made minor revisions to the survey and to the REDCap administration platform.

Table 3.

Team science competencies

Note: Scholar ratings less than or equal to 3.0 and TSE/E versus Scholar ratings greater than 1.0 are in bold.

The Return of Results process consists of the CTSA PI and the Director of the Clinical Scholars Program prereviewing the report to make sure that any negative comments are framed constructively. They then meet with the Scholar and the Scholars’ mentor to review the findings and focus on identifying areas where the Scholar thinks she or he needs additional training or opportunities to develop the competencies. The review also focuses on comparing the Scholars’ self-evaluation on each competency to the aggregate evaluation by the Scholar’s primary mentor and all of the TSE/Es. Where appropriate, the group identifies additional resources or programs that may help the Scholar develop the competency. After the meeting, Scholars are asked to complete a short anonymous survey (Table 4) providing feedback on the value of participating in the survey and the Return of Results process.

Table 4.

Survey for clinical scholars to report the value of receiving results from the team science competency survey

The first administration of the competency survey was in the Fall of 2019, followed by a second administration in the Fall of 2020, which included the incoming scholars of the 2020 cohort.

Statistical Analysis

Means and standard deviations were calculated for each item using the 0–5 scale, which spanned from “Not At All Well” to “Extremely Well,” respectively, so that a higher score indicated a greater sense of competence of the item. If the TSE/E or mentor evaluators did not have a basis for making a judgment, they selected “Not Applicable” and the result was set to missing. Equal weighted scales for each domain were calculated if greater than 50% of the items in the scale were non-missing. Data from Scholars and TSE/Es were then merged to calculate item differences as the mean of the combined scores of the TSE/Es and mentor minus the Scholar score so that a positive difference indicated that the TSE/E and mentor scores were higher than the Scholar’s score.

A Cronbach’s alpha was computed for each subscale to assess the internal consistency and inform reliability. To help better understand the scale structure, a factor analysis with principal component dimension extraction followed by orthogonal rotation was employed within each domain. Subsequent scree plots of the eigenvalues were examined to assess the dimensionality of the subscales. All analyses were conducted in SAS Studio V94.