Introduction
Translational research involves bridging between different stages of the translational spectrum (e.g. laboratory, clinic, community) to find, test, and implement solutions that address complex biomedical and health problems [Reference Austin1]. Innovative ideas are created by teams of multi-disciplinary researchers and stakeholders. There are only a few reports on the impact of scientific retreats [Reference Ranwala, Alberg, Brady, Obeid, Davis and Halushka2,Reference Ranwala, Alberg, Brady, Obeid, Davis and Halushka3], conferences, and other participatory activities [Reference Arevian, Bell and Kretzman4] designed to motivate and facilitate idea generation and team building. Despite conceptual advances in understanding determinants of successful translational collaboration [Reference Gilliland, White and Gee5], little has been done to intentionally create environments that facilitate development of translational ideas and creation of translational research teams, especially at the level of large research consortia.
The Un-conference is an innovative meeting that shifts from traditional didactic lectures and presentations to a more participatory format that engages attendees from multiple disciplines to co-develop ideas and form collaboration networks [Reference Greenhill and Wiebrands6]. Un-conferences facilitate cross-disciplinary idea generation, informal conversation, and formation of professional collaboration networks. Un-conferences differ from frameworks designed to arrive at group consensus among experts (e.g. Delphi Method [Reference Vernon7]), which are designed to arrive at a consensus about an issue, rather than form teams and collaborations to address an issue using novel approaches. The agendas for sessions and new collaborations emerge from unstructured, organic, interest-based, group discussions. This format has been highly successful within the technology sector [Reference Battelle8,Reference Ballinger9].
The University of Rochester Center for Leading Innovation and Collaboration (CLIC), the National coordinating center for the National Institutes of Health Clinical and Translational Science Awards (CTSA) Program, adapted the methodology of Un-conferences [Reference Greenhill and Wiebrands6] to develop Un-Meetings, which are designed to intentionally foster translational innovation, idea generation, and spark team collaborations, as previously been reported in detail [Reference Jones, Lane, Shah, Carter, Lackey and Kolb10]. Several Un-Meetings have been hosted by CLIC during last few years, which provided opportunities for idea generation and free conversations by multi-disciplinary panels of investigators and research users, on different translational topics, including healthcare workforce development [Reference Jones, Lane, Shah, Carter, Lackey and Kolb10], rural health and equity, artificial intelligence, life course research, academic-community hospital partnership development, and climate change [Reference Solway, Kenyon and Berglund11]. Here we use the Un-Meeting framework [Reference Sugarwala, White, Steele, Ossip and Zand12], a vehicle to support cross-disciplinary team formation and innovation, and a semantic network analysis method for studying these processes in the clinical and translational science community.
The unstructured format of the Un-Meeting allows researchers to form social clusters that may lead to early transdisciplinary team formation. In addition to formal organizational affiliations (e.g. institutions and departments), invisible communities of research and collaboration can form based on shared interests, values, and scientific worldviews that cross traditional organizational structures. These “invisible colleges” [Reference Zuccala13], “invisible communities” [Reference Tuire and Erno14], or “epistemic communities” [Reference Amin and Cohendet15] are difficult to identify and often are only nascent or yet-to-be-formed communities waiting for a catalytic event. However, they are critical in knowledge creation and formation of new disciplines [Reference Cohendet, Grandadam, Simon and Capdevila16]. Thus, identifying invisible research communities is critical for translational research and for addressing inherently multi-factorial and complex research problems (e.g. obesity [Reference Weng, Gallagher, Bales, Bakken and Ginsberg17], addiction [Reference Edwards18], or implementation science [Reference Estabrooks, Derksen and Winther19]) that span across biological, preclinical, clinical, public health, and socio-cultural realms.
The data collected through an Un-Meeting provide a unique opportunity to study the emergence of translational ideas for future research from the perspective of investigators interested in developing emerging collaborations, and to identify potential invisible research communities. Previous efforts identifying such ideas and communities have been largely limited to semantic analysis of translational research topics, heavily focused on bibliometric analysis of research outputs to identify hot topics and emerging fields (e.g. cancer research, heart disease [Reference Blümel, Gauch, Hendriks, Krüger and Reinhart20]) or common research approaches (e.g. epidemiology, pharmacokinetics [Reference Zhang, Wang and Diao21]). By focusing on the products of collaborations, such efforts miss the opportunity to identify cross-disciplinary potential teams, collaborations, and factors influencing team formation.
In contrast, analytic clustering of dynamic Un-Meeting topic keyword data allows a multifaceted analysis of attendee interests, dominant fields, and common frameworks. Such keyword analysis may identify bridging topics and cross-disciplinary approaches for future scientific team formation to address urgent research problems. Network analysis can identify topics that connect attendees from disparate research areas, similar to identifying bridging keywords in research publications that cross scientific fields [Reference Jaewoo and Woonsun22–Reference Cambrosio, Keating, Mercier, Lewison and Mogoutov24].
In this exploratory study, we describe content analysis of topics generated by participants in an Un-Meeting on the “opioid crisis” at University of Rochester. We investigated the clustering patterns and bridging action words, as indicators of dominant fields and bridging opportunities.
Methods
Un-Meeting Planning and Data Collection
The Center for Leading Innovation and Collaboration (CLIC), housed at the University of Rochester, hosted the first CTSA-wide “Un-Meeting” on June 2, 2018 addressing the opioid crisis. The Un-Meeting is a participatory event focused on engaging translational scientists, policy makers, community members, advocates, and public health professionals in a dynamic process of idea generation and translational conversation [25]. The Un-Meeting began with an introduction to the objectives and event rules, followed by a series of lightning talks (“4×4s”; 4 minutes, 4 slides) by scientists and decision-makers, aiming to provide a primer for potential topics related to the opioid crisis for subsequent small group breakout sessions [Reference Sugarwala, White, Steele, Ossip and Zand12].
After the talks, attendees used sticky notes to write topics of interest, questions, or ideas for breakout sessions and placed them on an idea generation board. Assisted by the organizing team, conference participants were encouraged to group the ideas into common themes that would determine the eventual topics for the subsequent concurrent small group breakout sessions. Facilitators and participants re-arrange and aggregate these notes into small group session topics in specific rooms and time slots. Attendees then self-select which sessions they would like to join. The resulting breakout sessions were 45 minutes long, unmoderated, and included between 8 and 25 participants. A second round of 4×4 talks, topic generation, and breakout sessions occurred in an afternoon session.
Topic Mapping
Photographs of the notes posted on the grid were used to collect the final clustering, and were transcribed into machine-readable format by hand. Each note was treated as a separate unit of text and was pre-processed by removing stop words (e.g. the, and, has) and common terms using the Automap program [Reference Carley, Columbus and Azoulay26]. The resulting keyword list was manually examined, and obvious synonyms were merged.
We used two analytical techniques to explore the patterns of topic clusterization:
Keyword cluster analysis: We clustered keywords based on their co-occurrence in notes and linked them with their corresponding breakout sessions using a density-based spatial clustering (DBSCAN) algorithm [Reference Ester, Kriegel, Sander and Xu27] with a centroid cluster dissimilarity function (Mathematica version 10.1; Wolfram, Indianapolis IN). The algorithm grouped together keywords on a two-dimensional surface based on commonality of their neighbors. The resulting clusters had keywords that were used together and shared similar neighboring keywords. In order to visualize the extent to which these clusters resemble the resulting breakout sessions, we developed a heatmap showing the commonality of keywords in clusters and breakout sessions.
Keyword-to-session network mapping: To identify keywords that bridged across fields and disciplines, we performed a separate analysis on keywords that were assigned to more than one breakout session. We expected that this analysis would provide clues to translational opportunities to address the opioid crisis and opportunities for cross-disciplinary team formation. We developed a keyword-to-session two-mode matrix and map, with nodes representing keywords and breakout sessions, and edges representing the relation between keywords and sessions. The nodes (keywords and sessions) were arranged in a two-dimensional graph using multi-dimensional scaling (MDS) of geodesic distances between nodes [Reference Jaworska and Chupetlovska-Anastasova28] and plotted using NetDraw [Reference Borgatti29].
Results
Participant Characteristics
In total, 102 individuals from 40 institutions (47% academic, 27% CTSA hub affiliate, 3% NIH, 11% other government agencies, 5% community organizations, 1% foundation, and 6% other) participated in the Un-Meeting. A third of participants came from within a 100-mile radius of the Un-Meeting location, the remainder represented 16 states and the District of Columbia.
Keyword Cluster Analysis
The goal of this analysis was to visualize the conceptual distance between and within breakout sessions. Such a mapping can show the conceptual diversity of the conference, grouping of words within the breakout sessions, provide an unsupervised clustering window into the breadth or focus of a particular breakout, and cross-connections between breakouts.
As a result of the idea generation session, 150 unique ideas were documented and grouped into 23 breakout sessions (Supplementary table 1). Unsupervised clustering of the 248 keywords in 150 sticky note ideas resulted in 18 clusters (Fig. 1A). The heatmap in Fig. 1B represents how frequently keywords within each cluster overlapped between breakout sessions. For example, the keywords in Cluster 4 were widely distributed across almost all sessions (Fig. 1B). This likely reflects the general medical theme of the keywords in this cluster, including ‘opioid’ (n = 39 occurrences), ‘treatment’ (n = 27), ‘pain’ (n = 15), ‘research’ (n = 8), ‘medication’ (n = 8), and ‘community’ (n = 8). Many less frequent, but conceptually distinct, keywords were also included in this cluster. Examples include ‘dissemination’ (n = 1), ‘outreach’ (n = 1), ‘workforce’ (n = 1), and ‘translate’ (n = 2). Similarly, cluster #2 keywords occurred broadly across 11 sessions, and included topics related to adoption/implementation/quality and evaluation: ‘access’ (n = 12), ‘implementation’ (n = 4), ‘context’ (n = 3), and ‘quality’ (n = 2). On the other hand, clusters with unique or limited multi-session occurrences included keywords such as ‘neonatal’ (n = 1), ‘prenatal’ (n = 1), ‘discrimination’ (n = 1), ‘disparity’ (n = 1), ‘racism’ (n = 1), ‘payment’ (n = 1), and ‘policy’ (n = 2).
Keywords cluster analysis. (A): the clustering map; (B): keyword cluster to breakout session heat map; (C): the list of keywords in each cluster.
We next examined the linkage between breakout sessions and keyword clusters. Breakout sessions also varied in terms of the breadth of the distribution of their keywords across clusters. Six sessions with highest between-cluster breadth shared keywords with five or six clusters. They included Disparities in Addiction, Access to Treatment (Session 17), Criminal Justice (Session 2), Clinical trials (Session 3), Community Engagement (Session 8), Special Populations (Session 11), and Health Insurance/Policy (Session 23). Most sessions addressed systemic and cross-disciplinary issues, involving community, governmental, and policy sectors.
Most interestingly, we found that the content of breakout sessions rarely corresponded with keyword clusters, as the content of clusters usually referred to multiple breakout sessions. Thus, clusters related to general medical aspects of addiction and systemic and organizational themes distributed across breakout sessions. Themes related to special populations and disparities did not appear in many sessions and were more specific to the sessions corresponding to these topics.
Keyword-to-Session Network Mapping
We next analyzed the relationship between keywords and sessions, based on their distribution in a network map, aiming to identify meaningful regions and distinct communities. Figure 2 shows the distribution of keywords (red circles) that were shared between sessions (blue diamonds) in the keyword-to-session network. As expected, given the MDS layout method, keywords shared by broader sessions concentrated around the center of the network map (e.g. “treatment,” “OUD,” “doctor”), while others that prominently bridged across various regions of the map appeared at the periphery.
The shared keyword-to-session map with MDS algorithm. Blue nodes: breakout sessions; red nodes: keywords; blue text: thematic regions; node size corresponds to the number of connections.
After reviewing, the common features of keyword clustered in different regions of the map and sessions that they bridged. We labeled seven main regions, denoted by the large blue labels in Fig. 2. The labeling process was a subjective activity to find labels that meaningfully describe the main themes of keywords and sessions in each region, and to identify thematic regions of the map. These regions were labeled as ‘community engagement’ (including keywords such as ‘engagement’, ‘partnership’, ‘network’), ‘technologies and innovations’ (including keywords such as ‘intervention’, ‘innovative’, ‘science’, ‘why’, and ‘methods’), ‘treatments and medications’ (including: ‘drug’, ‘pharmacology’, ‘development’, ‘prescribe’, ‘overdose’, ‘abuse’, ‘pain’), ‘clinical and best practice’ (including ‘guideline’, ‘clinical’, ‘best/better’, ‘practice’, ‘primary care’), ‘prevention and education’ (including ‘education’, ‘harm reduction’, ‘assist’), ‘special populations and disparities’ (including ‘recently-released’, ‘children’, ‘women’), and ‘system/context/complexity’ (including ‘structural’, ‘implementation’, ‘context’, ‘translate’, ‘population’, ‘data’).
In summary, we identified thematic regions in the map that can indicate to potential/invisible communities among researchers with common interests and approaches in opioid research. The main clusters included clinical research, translational research using technologies, population health, community engagement, special populations and disparities, and systemic and structural approaches to opioid disorders.
Discussion
In this report, we describe the use of unsupervised clustering based on semantic analysis of concepts and keywords to identify bridging concepts across expert domains and hidden communities of interest within an Un-Meeting involving investigators and research users interested addressing the opioid crisis. We used two complementary approaches to identify clusters of keywords and illustrated the overlap between sessions and keyword clusters, and areas that bridged across potential communities of research. We found that keywords related to treatment, pain, medication, access, and communities span across many breakout sessions that represented focused areas of investigation, while important concepts such as disparity, discrimination/racism, and policy were less frequent and were limited to few focused communities. The network analysis showed that focused communities were bridged by topics pertaining to the complexities of the opioid crisis, including systems, contexts, and community engagement, on one hand, and technologies, innovations, and treatment advancements on the other hand. These bridging concepts represent translational opportunities in opioid research that connect disciplines and research groups.
Results of the thematic analysis confirmed the conceptual diversity, with a majority of breakout sessions related to more than one topic area/disciplinary field, and spanning socio-political, technological, and clinical disciplines. This supports the translational nature of opioid research, similar to many other complex fields, in which, the expertise and research focus could not be easily classified by traditional departmental and disciplinary boundaries [Reference Andersen30]. This underscores the importance of identification and support of invisible/epistemic communities focusing on different aspects of this research, and development of strategies to bridge across them to facilitate knowledge translation and cross-disciplinary collaboration.
The opioid crisis affects several aspects of healthcare systems, spanning from prevention and public health to emergency care, and from primary care to specialties. The opioid crisis is even more complicated by a gradual change in the demographics of those most affected, and emergence of new behavioral trends [Reference Hohmann31,Reference Rouhani, Park, Morales, Green and Sherman32]. These complexities underscore the need for transdisciplinary collaboration between researchers and knowledge users (e.g. policy makers, community-based organizations, healthcare providers) at different stages of translational spectrum [Reference Humensky, Abedin and Muhammad33,Reference Jones, Varshneya, Hudzik and Huhn34], providing a much-needed and fertile opportunity for CTSA researchers and their institutions to promote translational scientific collaborations and innovations [Reference Cottler, Green, Pincus, McIntosh, Humensky and Brady35]. Major bridging topics identified in our analysis could be mapped to areas of current CTSA Program emphasis (e.g. community engagement, methods and processes, informatics, and clinical trials) [Reference Liverman, Schultz, Terry and Leshner36] and the emerging area of dissemination and implementation [Reference Dolor, Proctor, Stevens, Boone, Meissner and Baldwin37] and health equity [Reference Yousefi Nooraie, Kwan and Cohn38]. The bridging regions of the keyword map in our analysis are also consistent with the major areas of capacity building by CTSA hubs to address the opioid crisis, identified by Cottler et al. who classified them into preclinical, clinical research, community engagement, computational science and data informatics, and workforce development and implementation science [Reference Cottler, Green, Pincus, McIntosh, Humensky and Brady35]. While it is likely that this reflects the fact that over 75% of the conference attendees came from the CTSA consortium, this also provides an opportunity to identify potential areas for scientific team formation.
Inspection of the bridging concepts in the keyword-to-session map provides clues for translational opportunities in opioid research to be fostered by CTSA programs. These opportunities include:
-
Collaboration between clinical and basic sciences researchers to develop new medications and treatments. This is a dominant area in the center of Fig. 2, that involves themes related to innovative treatments, use of technologies to develop innovative methods of research and evaluation, and data science.
-
Collaboration between health researchers and community members as well as community-based institutions, to engage stakeholders in research and to address disparities. The Community engagement region in Fig. 2 connects themes related to treatment, prevention, education.
-
Collaboration between clinical researchers, system scientists, dissemination & implementation scientists, and sociologists to study the complexities of social systems and organization of care. The region labeled System/Context/Complexity in Fig. 2 is located adjacent to Special populations/Disparities, which underscores the role of systemic approaches to improve health equity in this area.
While the results of our study at a minimum provide a potential roadmap of opportunities for translational advancement in opioid research among this core subgroup of scientists, there are several limitations to be acknowledged. The keywords used in the mapping are based on limited content provided on sticky notes and might miss the complexities and breadth of subject matters to which they correspond. This may affect the richness of conceptual clusters and resulting interpretations. The assignment of topics to breakout sessions was moderated by a facilitator and hence does not reflect a completely unsupervised process of clustering. The topics also reflect the areas of expertise of this particular group of attendees and may not be generalizable. Nevertheless, this method of semantic analysis with thematic network analysis and keyword clustering may be useful for analyzing Un-Meetings with the aim of identifying hidden communities, areas of potential collaboration, and potential gaps in a research network’s ability to impact a chosen translational research problem. The value of these clustering techniques in identifying emerging communities and future collaboration potentials could be assessed using mixed methods studies to incorporate the perspectives and reflections of investigators involved in idea generation activities into the analysis.
Conclusions
Our case study of mapping the generated ideas in an Un-Meeting on the opioid crisis, an inherently translational issue, provided insights on the thematic clusters and topic areas that bridge across them. Our findings add to the scientific body of literature on using thematic mapping to reveal the complexities of inter-disciplinary research [Reference Wagner, Roessner and Bobb39] and provide clues about emergent themes to inform strategic planning in translational research [Reference Falk-Krzesinski, Contractor and Fiore40]. The dynamic process of grouping breakout session ideas into clusters is similar to other graphical approaches used to classify concepts and topics, such as Concept Mapping [Reference Falk-Krzesinski, Contractor and Fiore40–Reference Trochim and Kane42]. The Un-Meeting format appears particularly suitable to study the dynamics of idea generation and team building in translational science, where transdisciplinary teams are critical to advance novel discoveries and innovative solutions to biomedical problems across multiple “translational barriers” via cross-disciplinary teams [Reference Gilliland, White and Gee5,Reference Sohn43].
The cross-disciplinary nature of topic areas that bridge across thematic communities provide opportunities for CTSA programs to engage and support diverse communities of researchers and users to develop translational teams. Potential opportunities to support translational collaborations include technological advancements of opioid prevention, treatment, and surveillance; studying systems, contexts, and complexities of the opioid crisis, and studies focusing on special populations and health disparities.
Supplementary material
To view supplementary material for this article, please visit https://doi.org/10.1017/cts.2022.490
Acknowledgements
The authors would like to thank Jeanne Holden-Wiltse, Raquel Ruiz, Melissa Trayhan, and Kate Fetherston from CLIC for their assistance with the Un-Meeting.
This work was funded in part by the University of Rochester Center for Leading Innovation and Collaboration (CLIC), under Grant U24TR002260 and the University of Rochester CTSA award number UL1 TR002001, both from the National Center for Advancing Translational Sciences of the National Institutes of Health. This work is solely the responsibility of the authors and does not necessarily represent the official views of the NIH.
Disclosures
None of the authors have any actual, potential, or perceived conflicts of interest to disclose.
Open access
