Introduction
A glance at the headlines over the past year of the pandemic confirms the extraordinary importance of biomedical innovation. Safe and effective drugs, vaccines, devices, and diagnostic assays, which might have required years or decades to develop in the past, have been introduced in a fraction of that time, saving countless lives from the ongoing pandemic [Reference Fauci1]. Such breakthroughs are not limited to emerging infections and have been realized in the form of comparable advancements for targeting cancer by redirecting the immune response, robotic devices to assist surgery, and genomic analyses to identify, prevent, and, soon, to intervene against genetic maladies [Reference Shivram, Cress, Knott and Doudna2–Reference Sharma, Siddiqui and Anandhan4].
The breakthroughs attributable to biotechnology have not merely improved public health but become critical contributors to the economies of most developed nations, yielding high-paying jobs and tax bases. However, a continued ability to develop such breakthroughs is increasingly uncertain. This concern has perhaps been most well-examined in terms of pharmaceutical development. Long-term analyses reveal that the efficiency in discovering and developing new medicines has been declining at an exponential rate since at least the 1950s [Reference Munos5]. Termed “Eroom’s Law” (a playful inversion of the well-known Moore’s Law of Computing), these growing inefficiencies may soon threaten our ability to introduce new medicines altogether [Reference Scannell, Blanckley, Boldon and Warrington6]. The responses taken by the private sector to break Eroom’s Law have included consolidation and outsourcing, both of which have paradoxically ensured growing inefficiency (and thus further adherence to Eroom’s Law). Whereas pharmaceutical companies had historically conducted much of the discovery and research of new medicines, these activities were relegated to upstart biotechnology companies (contributing to the biotechnology revolution that began in the 1970s) and eventually also to academic organizations. Indeed, our recent findings have demonstrated a growing reliance upon academic inventors as crucial sources for discovering FDA-approved medicines [Reference Kinch, Horn, Kraft and Schwartz7].
Despite this growing reliance upon academia, it is widely understood that most academics, and their home institutions, have neither the experience nor infrastructure to reliably conduct efficient discovery and development of medical products. By definition, these organizations focus upon teaching and fundamental research. Nevertheless, many institutions have recognized this increased need for supporting technology commercialization, some of which have established entire centers for this express purpose [Reference Gulbranson and Audretsch8,Reference Bradley, Hayter and Link9]. While gains have been made, especially among the pioneers in academic translation in and around Silicon Valley (e.g., Stanford) and Boston (e.g., MIT), most universities have neither the pedigree, location, nor resources to build such capabilities. Therefore, new approaches are needed to help translate the practical knowledge and practices required for entrepreneurial success in biotechnology. This led us to ask how other academic centers might leverage their internal expertise to help discover and develop biomedical breakthroughs.
One common method to promote these endeavors within academic centers is to provide grants focusing on entrepreneurial or commercialization activities [Reference Munari, Sobrero and Toschi10]. Often termed university gap funds or proof of concept programs, their format ranges from more traditional written applications to accelerator-style pitch competitions and often contain elements of both. Thus, our team at Washington University in St Louis (WUSTL) evaluated similar mechanisms at the university that helped identify and advance promising academic projects into commercial opportunities.
Academic Entrepreneurship at Washington University in St. Louis
Although, and perhaps because, WUSTL had excelled as an academic center focused upon basic sciences, the university was not highly ranked in terms of commercial licenses and was in the bottom decile in start-up formation from fiscal year 2010–2015 (per $10M research funding) according to data from the Association of University Technology Managers Licensing Activity Survey [Reference Allard, Miner, Ritter, Stark and Stevens11]. In discussing this fact with key research faculty, we soon learned that while the vast majority of academic leaders at the university were interested in entrepreneurship, they did not have the time to learn how to become successful entrepreneurs. Consistent with this, recent surveys suggest that academic investigators are now required to expend roughly 40% of their time writing and maintaining grants, which severely impinges upon their ability to develop new skills [12]. Therefore, we sought to create the means to help create a sustainable entrepreneurial environment while minimizing the time requirements needed by successful scientists, who were already over-committed and yet under-performing in terms of innovation and entrepreneurship.
Since the early 2000s, WUSTL had hosted a 2-minute pitch competition for its faculty, termed the Bear Cub competition. The objectives of this program, at a glance, aligned with those of university gap funds in that they focused upon the commercialization potential of university technologies. However, our internal experiences suggested that the impact of the Bear Cub competition fell short of achieving these goals. The competition’s “elevator pitch” format, rather than emphasizing substantive discussion of scientific promises and pitfalls, tended instead to reward charismatic presenters or projects. This trend became starkly apparent after the program began offering a slide design service to teams. Though the intent was to allow investigators more time to focus on the commercialization aspects of their projects, we later discovered that the utilization of this service, independent of scientific quality or entrepreneurial value, became one of the most significant predictors of funding success. We also found that Bear Cub winners frequently moved on to other, similar competitions after winning instead of focusing on the advancement of their funded project.
In addition to our observations, faculty members expressed concerns that the value provided by the program was disproportionate to the time commitment required to participate, which included several meetings with the program organizers focused on teaching investigators how to perfect their pitch. Rather than focusing on the relatively narrow skill of pruning a talk to fit the required format, these investigators sought an engagement that clarified the priorities of technology commercialization (compared to basic or clinical research). They also desired recognition that each project is unique with different opportunities and challenges and thus requires more individualized feedback and support.
In this study, we describe recent efforts to promote academic entrepreneurship at WUSTL through the conversion of Bear Cub into a gap fund program that addressed several limitations of the “elevator pitch” competition. That is, the relatively shallow nature of the 2-minute pitch and the poor return on investment for participants. This program, termed LEAP, was housed under the university’s entrepreneurship center (the Skandalaris Center for Interdisciplinary Innovation and Entrepreneurship) [13] and encompassed a series of changes intended to cultivate an understanding of commercialization among the WUSTL scientific community. After outlining our methodology, we describe these changes in detail, followed by analyzing both participation and survey data to assess the cultural impact of the program. Lastly, we examine WUSTL start-up formation to estimate LEAP’s role in supporting technology commercialization at the university. Ultimately, our goal was to craft a sustainable culture of innovation and entrepreneurship.
Methods
LEAP Program Structure
Application Process
To address the pitfalls of Bear Cub, we started by modifying the framework of a conventional business plan competition and created a program that allowed investigators to enter with minimal up-front requirements. First, we allowed investigators to apply based upon a brief description that emphasized a potential partnering opportunity for their technology. The only requirement was that it must be based on potential WUSTL intellectual property to be submitted as an invention disclosure to the WUSTL Office of Technology Management (OTM). This simple action would initiate a guided process intended to add value to projects, regardless of whether they ultimately received funding. Rather than relying solely upon faculty members, we asked that projects be represented by teams that included pre- and postdoctoral trainees, staff, and other faculty members as available. Our rationale was that teams, especially those with early-career individuals, would be more likely to convey learnings internally and to their colleagues, who might themselves become interested in entrepreneurial endeavors.
Orientation and Background Questionnaire
After applying, teams were informed of specific requirements for each stage and received reminders of necessary deliverables throughout the cycle. This orientation increased the potential that the teams would maintain momentum in crafting their project and pitch. We also provided a background questionnaire based upon standard business plan frameworks and asked each team to answer this series of basic questions. This document served as the final evaluation project summary, so each team was instructed to develop the document throughout the cycle.
Program Personnel
Recognizing that much of the requested information would be utterly foreign or likely to be misunderstood, a staff of part-time associates and volunteers (often graduate/MBA students, postdocs, alumni with relevant expertise, and other university contacts) was curated to help guide the applicants and field questions throughout the cycle. These individuals also helped prepare written summaries of feedback and action items for the teams after each meeting. From the standpoint of staff infrastructure, the entire enterprise did not exceed 1.5 full-time employees at any given time.
LEAP Meetings and Final Evaluation
As the goal was to impart new thinking and skills to the team (and thus the larger environment), focused, one-on-one interactions would prove essential. Thus, we opted to hold two individual meetings with each team, each attended by the LEAP program lead, relevant industry experts (if available), an OTM staff member, and a trainee for capturing written feedback. The first was an informal meeting to discuss the project and its potential. These meetings were purposely held without a lengthy introduction and often without slides since these tended too often to cause the teams to revert to a conventional academic discussion. Instead, questioning emphasized how the intended product might improve the current standards and other key project aspects concerning intellectual property, potential partners, resource needs, and milestones to define success and failure. The teams were then expected to discuss these questions, make any necessary changes to their project strategy, and return roughly 6–8 weeks later for their second meeting.
The second meeting was designed to emulate the structured final evaluation by a panel of judges at the end of the cycle. Rather than relying upon a conventional elevator pitch, teams were instructed to convey the opportunity (including its uniqueness and resources requested) in a 15-minute presentation. The presenters would then be questioned for up to an additional 30 minutes, leaving 15 minutes for judges to discuss and score the presentations. In the practice sessions, the remaining time was instead used to summarize presentation feedback and distill any action items the team needed to address. The teams were then given an additional 6–8 weeks to address these improvements before the final evaluation.
The final evaluation judges were generally vice-president level individuals from relevant industries (e.g., pharmaceuticals, medical devices, venture capital). They were segmented into panels matching their expertise and then instructed to recommend projects that met two chief criteria: 1) the project, if completed, would be likely to be partnered, and 2) funding from the program would substantially increase the potential for partnering. The judges were allowed discretion in suggesting modifications to the proposed milestones (i.e., eliminating unnecessary work or identifying potential “fatal flaws”) or budgets, the latter of which was capped at a level not to exceed $50,000. Judges were asked to include a rationale for their decision and any feedback they wished to provide directly to the teams. Similar to the previous meetings, feedback was compiled and distributed to each team after the event. Winners were then selected from teams with the highest recommendations, with awards to be disbursed in tranches upon completion of each milestone.
Program Costs
The staffing costs for the program were kept to a minimum and included the LEAP program manager and the part-time associates. As mentioned earlier, this amounted to no more than 1.5 dedicated FTE ($165,000 for salary and fringes annually). A similar level of expenditure had been applied to the earlier program and a major improvement included the use of alumni and other friends of the University to provide support and to serve as judges. Average total award cost was approximately $132k per cycle but would vary depending on the award number and amount. Contributions from funding partners (other University entities with special interests, such as cancer centers) doubled the total award allocations to approximately $269k per cycle. Other costs included renting the rooms, food, parking, and other requirements needed to run the LEAP Final Evaluation events (∼$6700 per cycle), as well as other miscellaneous LEAP-related costs ($3000 annually) were experienced. Since the program began conducting reviews and the Final Evaluation virtually via Zoom (a consequence of COVID-19), most of these event costs have been eliminated.
Data Collection and Analysis
Post-LEAP Sentiment and Knowledge Transfer Survey
Following the LEAP Final Evaluation event, LEAP principal investigators (PI) were asked to complete a feedback survey administered using Qualtrics (https://www.qualtrics.com/). To evaluate sentiment, respondents rated their agreement with statements about the LEAP program pertaining to the value provided and program logistics. Possible responses ranged from “Strongly agree” to “Strongly disagree” using a 5-point Likert scale. To evaluate knowledge transfer, respondents self-rated their knowledge in seven areas related to technology commercialization before and after their participation. Possible responses ranged from “Extremely knowledgeable” to “Not knowledgeable at all” using a 5-point Likert scale.
LEAP Participation and Funding Rates
Most project data were collected from the LEAP database, an operational and archival tool for program information. Initial project data, including title, description, and team members, were sourced from intake forms. Projects were recorded as “Presented” only if they participated in the final evaluation and were otherwise recorded as “Dropped.” The funding status of each project was documented at the end of each cycle.
For the repeat PI participation and funding rate analysis, projects were designated as “New” or “Repeat” based on whether the project PI had previously led a “Presented” LEAP project. Restricting this analysis to only “Presented” projects ensured that the level of engagement with the program was uniform for each included project. Engagement level was not feasible to determine for “Dropped” projects, as that designation was used regardless of when the project ceased participation. For example, a project that left the program immediately after applying and one that did just before the final evaluation would both be designated as “Dropped.” Additionally, since only “Presented” projects are considered for funding, it was necessary to restrict the funding rate analysis to that group.
Start-Up Formation and LEAP-Supported Agreements
To assess start-up formation, we calculated the average annual rate of start-up license agreements executed during a given fiscal year range. These records were sourced from the WUSTL OTM’s database powered by InnovateIP (recently acquired by Cayuse). For the purposes of this study, each start-up agreement represents the formation of a newly created venture based on WUSTL intellectual property.
To assess the relationship between LEAP and WUSTL start-up formation, LEAP team members and project technologies were cross-referenced with the OTM start-up agreement records. In brief, a start-up agreement was considered LEAP-associated if the agreement PI had also been a PI or team leader on a LEAP project before the execution of the agreement.
Results
The program was held twice per year to remain timely (vs. once per year, as many gap fund programs tend to do). This modification was intended to increase the likelihood of promoting an entrepreneurial culture, as it allowed investigators to be either actively participating in the program or preparing to do so. Despite this increased frequency, participation remained steady from the initiation of these changes in mid-2015 through the end of 2020 (Fig. 1A). We also found that whereas the two-minute pitch competition tended to draw from the same pool of participants, this new program consistently attracted new ones. In nearly all cycles of the modified program, more than half of the presenting teams were led by investigators who had not participated previously (Fig. 1B), which addressed the aim of expanding the number of trained teams and thus helped foster the creation of an innovative culture.
LEAP participation rates show consistent participation of new investigators. (A) Count of participating teams in each cycle, classified by presented and dropped statuses. From top to bottom, data labels on each column represent the number of total teams, dropped teams, and presented teams, respectively. (B) Proportion of presented teams led by new and repeat principal investigators in each cycle. From top to bottom, data labels on each column represent the number of repeat teams and new teams, respectively. PI = principal investigator; SP = Spring; SU = Summer; FA = Fall.
We also questioned whether prior investigators had benefited from participation. For most cycles, teams led by repeat investigators were more likely to receive funding than those led by new investigators, suggesting that prior training had proved useful (Fig. 2A, B). Notably, approximately a third of successful repeat investigators had applied to the program with a new scope of work (13/38), suggesting that their prior experiences may have helped them select a more relevant project.
LEAP participation shows returns on investment through an increased funding rate for repeat investigators. (A) Funding rates of presented teams led by new and repeat investigators by cycle. (B) Box plot depicting the funding rate distribution of presented teams led by new and repeat investigators. Excludes the Spring 2015 cycle, as there were no repeat investigators. Crosses represent distribution means. Small circles represent individual data points. PI = principal investigator; SP = Spring; SU = Summer; FA = Fall.
All presenters were asked to complete a post-cycle survey to refine the program and assess its impact. In one section, respondents were asked to rate their agreement with a series of statements about their participation. Upon analyzing responses from several cycles, we were surprised to find that more than 90% of respondents agreed with the statement that the time spent preparing for LEAP was worth the investment, regardless of funding outcome (Fig. 3A). In the same survey, respondents were asked to self-rate their understanding in several fundamental areas required for entrepreneurial success (e.g., intellectual property, regulatory strategies, external partner expectations) before and after participation. In almost every case, investigators expressed that they had a greater understanding of these areas after completing the training than before entering the program (Fig. 3B).
LEAP participation is highly valued by investigators and leads to increased self-rated commercialization competencies. (A) Breakdown of investigators’ level of agreement with the statement “Time spent preparing for LEAP was worth the investment, regardless of funding outcome.” Data collected from the Spring 2019 cycle onwards. n = 66 (B) Investigators’ average self-ratings in seven key areas necessary for successful academic entrepreneurship before and after LEAP training, with 5 being “Extremely knowledgeable” and 1 being “Not knowledgeable at all.” Data collected from the Fall 2018 cycle onwards. n = 84 (83 for Potential Customers/Commercialization Pathway). SP = Spring; FA = Fall; BEFORE = Before LEAP training; AFTER = After LEAP training.
Perhaps most importantly, we asked whether we could assess the impact that LEAP had on the state of WUSTL commercialization. For this, we investigated the number of start-up licensing agreements (synonymous with start-up formations) that were executed during the fiscal years immediately before and after implementing the LEAP program. During the Bear Cub years (FY03–14), we found that the institution had participated in an average of 2.75 start-up agreements annually (Fig. 4A). In contrast, the average annual rate had more than doubled in the years following LEAP’s adoption (FY15–20) (Fig. 4A). We then asked how many of these agreements LEAP had supported, either directly through involvement with the project or indirectly through contribution to the university’s shift toward an entrepreneurial culture. To do so, we looked at whether the PI on each agreement had previously been a PI or team leader on a LEAP project. Indeed, more than two-thirds of the start-up agreements fulfilled these criteria (Fig. 4B).
Implementation of the LEAP program correlates with increased start-up formation. (A) Average annual number of start-up formations during the periods where either Bear Cub (FY03–14) or LEAP (FY15–20) was active. Error bars represent standard error of the mean. (B) Portion of WUSTL start-up formations from FY15–20 that were supported by LEAP (LEAP) or were not supported by LEAP (Non-LEAP). FY = fiscal year.
Discussion
Our results suggest that the creation and improvements in LEAP contributed to this turnaround in start-up formation at WUSTL. Importantly, these improvements were implemented without dramatically increasing the magnitude of funding or expanding the infrastructure meant to promote innovation. Instead, our goal was to foster an entrepreneurial environment that could help the institution better leverage existing resources and expertise. Several of our findings support LEAP’s role in this cultural shift, including the sustained proportion of new investigators each cycle, the increased funding rate for trained investigators, and the improved self-rated competencies. These results, along with the high representation of LEAP-associated start-up agreements, indicate that a LEAP’s practical approach to individualized, team-based training (as opposed to didactic lecturing) can successfully impart a cultural shift onto a willing audience and thus contribute to sustained improvements in commercialization outcomes.
Beyond the programmatic changes, LEAP’s rehousing under the WUSTL Skandalaris Center likely also played a role in this cultural shift. Teams participating in LEAP were encouraged to engage with other programs and resources that the center offers. For example, the center’s Expert on Call and In-Residence programs provide access to experts in various subject domains. Such engagement served to draw team members (especially faculty) into the larger entrepreneurship community at the university, thus increasing the likelihood that they would stay engaged in academic entrepreneurship beyond their participation in LEAP.
One unexpected side effect of LEAP’s launch was the interest it drew from potential funding partners at the university, several of which were focused on particular project types. In recent years, at least two funded projects per cycle have had their awards backed, partially or in full, by one of these organizations. We posit that much of this interest stems from the increased scientific rigor in the modified LEAP pipeline, allowing funding partners to contribute to validated projects without the need to maintain the pipeline infrastructure themselves.
Notably, one of these partners, the Center for Drug Discovery (CDD), was founded in part because there was interest in funding more pharmaceutical projects through the LEAP pipeline. In addition to funding projects for the original $50,000, the CDD partnership has also given rise to additional awards for exceptional drug discovery projects, up to a total of $100,000. Other LEAP funding partners include the Institute of Clinical and Translational Sciences, the Siteman Cancer Center, and the McKelvey School of Engineering. This example illustrates that programs like LEAP can not only serve as a nexus for university centers interested in supporting academic entrepreneurship but also serve as a nucleation point for new ones.
Comparison to Other Programs
LEAP’s support of university commercialization through funding and expert guidance aligns with other gap funds and proof of concept programs described in the literature [Reference Gulbranson and Audretsch8,Reference Bradley, Hayter and Link9,Reference Gubitta, Tognazzo and Destro14–Reference Battaglia, Paolucci and Ughetto16]. Consistent with our results, previous studies have also noted an increase in start-up formation after the introduction of these programs [Reference Bradley, Hayter and Link9,Reference Hayter and Link17]. Notably, we found that most studies did not include details on the structure of individual programs, making it difficult to evaluate the level of mentoring and feedback provided to participants. Nevertheless, the type of entrepreneurial education and team-based engagement that LEAP promotes has been increasingly recognized as being important for efficient university tech transfer [Reference Battaglia, Paolucci and Ughetto16,Reference Boh, De-Haan and Strom18–Reference Duval-Couetil, Ladisch and Yi20].
Limitations
Institutions interested in starting a gap fund akin to LEAP should be mindful of limitations intrinsic to this lean, team-based training model. First, the one-on-one interactions necessitate that the number of teams is limited (either at the outset or filtered mid-cycle) to allow time for meetings and ensure that the program lead can devote sufficient attention to each team. Second, LEAP heavily relies on a diverse network of volunteers. While some aspects of the program can be managed without them (e.g., the program lead prepares written feedback when no trainee volunteers are available), a lack of knowledgeable professionals to provide feedback at meetings and, particularly, the final evaluation would limit the value provided by the program. Interested institutions can address this by utilizing their alumni network for the initial judge cohorts. Finally, for institutions with fewer resources, the $50,000 award amount may serve as a barrier. However, we find that a successful cycle can still be run with reduced award sizes, given that the funds are only a part of the value that the LEAP provides.
Conclusion
In addition to LEAP’s contributions, the increase in licensing we observed was undoubtedly influenced by other improvements in the region. Among these is an expanded integration of the robust St. Louis ecosystem with university programs and better research community engagement through conventional technology transfer efforts. Nevertheless, these findings suggest that training programs such as LEAP represent an opportunity for promoting translation and inculcating entrepreneurship in environments that have historically been underperforming. Although the LEAP program is one contributor to the improvement we observed at WUSTL, these results show that a comparatively modest investment, matched with a targeted and individualized approach, can yield disproportionately positive outcomes.
Acknowledgments
We thank II Luscri, Mike Seper, and the rest of the Skandalaris Center team for facilitating the development and execution of the LEAP program. We thank Nichole Mercier from the WUSTL Office of Technology Management and II Luscri for guidance in drafting this manuscript. Research reported in this publication was supported by the National Center For Advancing Translational Sciences of the National Institutes of Health under Award Number TL1TR002344. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
Disclosures
The authors have no conflicts of interest to declare.
Open access
