OCA strengths

Participants indicated that the OCA’s individual DOD CDMRP OCRP-sponsored funding (‘ECI Awards’) were helpful to their overall development because it allowed participants to focus on building a program of research without having to worry about funding. As one participant explained, “A lot of funding opportunities are much shorter, which makes it really hard to kind of settle in” (Participant3). Participants indicated that having a funded laboratory allowed them to refine their research goals, identify potential grants, and kick start their nascent research programs. Funding also helped expand their laboratory capacity, as this participant explained, “The funding has definitely been instrumental in shaping my lab, getting things done, and getting students” (Participant5). Participants also explained that not having to worry about securing additional funding for a period of time allowed them to focus on their own professional development as scientists. Mentors corroborated this finding indicating that many of the junior awards available to researchers are only for 2–3 years making the OCA funding incredibly helpful, “My mentee was able to write one small grant application and then be able to get [a] five year grant, which really made all the difference in terms of the successful career that he has now” (Participant9).

Additionally, being able to develop as a professional was an unanticipated benefit. Having reduced pressure to secure additional funding meant they could focus on the continued development of their research program and build a support network that would endure long after their participation in the program ended. For example, this participant explained, “I think that [what was most valuable] was both the funding aspect of it as an early career investigator and also the community aspect… I didn’t realize how strong the community would be at the time I was applying” (Participant3). Having a supportive community was important because it reduced isolation, but also provided technical support when participants needed it. One participant described it as “a family where we help each other” (Participant8), suggesting that both personal and professional relationships were created. Mentors also thought the community building was important because it created a safe and collaborative atmosphere for participants, which is a departure from other programs or scientific meetings where attendees compete with each other. In the OCA, there was no need for competition because everyone was there for the same purpose, to support the work of ECIs in ovarian cancer research, as one mentor explained, “The Academy is very ideal because everybody gets along so well. I mean, you see very little friction in the Academy meetings… [because] they’re all in the same mode there” (Participant12).

Participants also liked that while participating in the program their success was tracked, which helped them to reflect on what they have accomplished throughout the year and make plans for the following year. For example, one participant indicated she liked that the OCA tracked the grants she applied for, including ones that were funded and ones that were turned down. She also liked that the OCA was interested in her publications, committee work, and her other professional development activities. Again, this gave her a sense that there were others in her community tracking her development who could help her think through areas she may need to strengthen as an ECI moving toward independence.

Additionally, the ECIs problem-solved everyday issues in dealing with their institutions or departments, as this one shared, “[The OCA was valuable] to not just connect with [others] scientifically, but [to hear about] the nuts and bolts of how they were doing within their own institution’s department.” Another participant described how having a community was useful to quickly move through issues in ways that have the potential to impact productivity. Participating in a virtual CoP normalized communication in ways that were not available face to face, “We talk frequently [via] a chat room. Whoever has problems, technical issues, or you know, when we applied for grants, we face difficult questions, and so we talk to each other [about it]” (Participant8).

Finally, participants described the high-quality mentoring they received on how to conduct ovarian cancer research. As one participant explained, “People don’t really realize how much it means for us to have that kind of support and mentorship in place” (Participant6). Mentors took their roles seriously and spent considerable time supporting mentees in their research. The protected time they received from being a co- or sub-PI on the ECI’s grants helped ensure mentors could focus on the needs of the mentee, as this mentor explained, “It is nice to have a little bit of protected time, I take my mentoring very seriously and so I meet with all my mentees. I would say that 25% of my week right now is [spent mentoring]” (Participant13).

ECIs also expressed that they grew as mentors themselves. For example, one participant indicated that through the mentoring relationship they learned about the importance of being a team player. Whereas previously he did not take his role on a team as seriously as a leadership role, the OCA helped him realize the importance of this, “Even when you are not the leader of the group, you need to be a good team player [and] you need to care about the team” (Participant1). Others described learning about their responsibility to others in different ways. For example, another participant shared, “[The OCA] really taught me more about mentoring, how to become a better mentor, how to navigate interactions with mentees, and how to get the most out of interactions with mentors” (Participant6).

In summary, the OCA’s first cohort of ECIs to be evaluated found that the program’s funding, networking and community building, and also expert mentoring, were extremely helpful to their growth and development as ECIs in the field of ovarian cancer research. These results were substantiated by the participants’ mentors who felt that the program was successful in helping participants successfully launch themselves in the early phases of their research careers, as this one indicated, “The success rate is going to be pretty substantial, and I think it speaks to the role of the Academy in helping foster the early phases of career development for those individuals” (Participant11).

OCA areas of improvement

Although participants felt there were many areas of strength, a subtler theme showed the need for more skill development on grant writing. ECIs indicated that what was provided in the OCA was very helpful, as this one expressed, “the grant writing and grant review process is particularly strong” (Participant3). However, the mentors and mentees expressed concern that once they left the program, they wouldn’t be able to secure their own funding, even though several had already secured additional funding at the time of the exit interviews. They also expressed that conducting team science was challenging, given that they had their individual research projects they needed to attend.

Additionally, the inclusion of patient advocates in the OCA was perceived differently by physician vs. non-physician scientists. Some felt that these interactions were exceptionally rewarding because this was an aspect of ovarian cancer research for which they had little previous experience, such as this one, “We got a lot of opportunities to interact with consumers and patient advocates, which generally we don’t get. The OCA was a very good platform to get to know the ovarian cancer patients and what they went through” (Participant7). Mentors also felt that this was a benefit, explaining, “I’ve been in ovarian research for 15 years and actively involved in a number of meetings addressing ovarian cancer …to actually see the advocates, I think has been extremely valuable” (Participant12). However, other ECIs indicated they would like even more regular opportunities to interact with these community members because of how powerful their stories were in shaping their research, as this ECI shared, “I kind of want more interaction with the patients and patient advocates” (Participant6).

In summary, after participating in the OCA, participants thought that more opportunities for grant writing and grantsmanship development, with potentially even greater inclusion of patient advocates in activities of the program would be useful. Although these components were already included in the design of the OCA, participants felt they were beneficial to their career development and wanted even more opportunities to grow and learn from such experiences.

Survey data

Internal reliability was calculated using Cronbach’s Alphas. Table 3 presents the reliability for each scale (using Time 1 data) and the means and standard deviations from for Time 1 and Time 2 data for each assessment. The minimum and maximum scores for each item and each scale, as well as the number of items per scale can be found in Table 2. Because the program satisfaction items assess different aspects of the program, item reliability is not calculated as these were used at the individual item level.

Table 3.

Comparison of means for mentees at time 1 and time 2

Note: CI, confidence interval; LL, lower limit; UL, upper limit.

* These scales showed a notable change of .5 scale points or greater between Time 1 and Time 2.

⁺ These scales show a P < .10, indicative of a trend toward significance.

Because we used combined time points as detailed in the analysis plan, there was no missing data in Time 1 and Time 2 comparisons. Still, no statistical significance (P < 0.05) occurred between Time 1 and Time 2. This was anticipated given the small n for the first year. However, many measures displayed an increase in mean scores between Time 1 and Time 2 in the directions predicted. Table 3 summarizes the difference in scores between Time 1 and Time 2. In particular, we noted scales in which there was a .50 point or greater change. While not statistically significant, investigators, specifically the principal investigator and two evaluation team members, reached a consensus that this was a meaningful change worthy of further consideration. Table 4 presents the individual item data on program satisfaction.

Table 4.

Evaluation by mentees of the Ovarian Cancer Academy (OCA) programs components

Note: Items were scored on a 7-point scale, where 1 = Not At All Helpful and 7 = Extremely Helpful.

Publications and grant data

The total number of publications produced by the first OCA mentee cohort to be evaluated for the period 2015–2020 was 109, 60 of which an ECI served as the first or last author. The total number of NIH grants received was 7, accounting for 2.5 million total dollars.