Literature mapping is a powerful method for analyzing a particular field’s academic landscape. In recent years, comprehensive reviews of scientific fields have become increasingly challenging because the number of publications has grown beyond human cognitive capacities (Wagner, Lukyanenko, and Paré Reference Wagner, Lukyanenko and Paré2022, 209). Traditional literature reviews, which rely on the researchers’ subjective judgment, have generated problems such as in-group citation practices (Zhou, Chai, and Freeman Reference Zhou, Chai and Freeman2024), citation bias (Esarey and Wu Reference Esarey and Wu2016), and reliance on conventional indicators of scholarly impact (Ramírez-Castañeda Reference Ramírez-Castañeda2020). In addition, many researchers no longer attempt to create comprehensive reviews of the literature (Knopf Reference Knopf2006).
Statistical bibliometric methods have been used to remedy some of these difficulties. Such methods include systematic search and analysis techniques, as well as visualization tools (Van Eck and Waltman Reference Van Eck, Waltman, Ding, Rousseau and Wolfram2014). Visualizing a scientific field’s universe of publications enables scholars to generate clear images of networks that communicate complex information in a simple form and provides comprehensive coverage of research fields, reducing the level of subjective judgment (Booth-Tobin et al. Reference Booth-Tobin, Munis, Smithson-Stanley and Han2021). In addition to analyzing a list of topics or cited authors, literature mapping analyzes the relationships between them. Units of analysis include research areas, scholars, institutions, journals, and countries where research originated. Literature mapping is therefore ideally suited to answer critical questions such as the following:
-
1. What areas of research exist within a scholarly field? How well connected are these areas? Answering these questions helps researchers achieve objectives such as identifying areas where synergy is needed (Adro and Fernandes Reference Adro and Fernandes2022) and understudied areas (Park et al. Reference Park, Montecchi, Plangger and Pitt2020), tracing the evolution of a field (Fils and van Eck 2018), and identifying emerging trends (Goncalves et al. Reference Goncalves, Kieckbusch, Perna, Fujimoto, Morales and Romanelli2019).
-
2. What studies form the canon of a field? What are their main themes, and what is the relationship between them? Answering these questions enables scholars to identify the studies that form a research area’s canon in a rigorous way, to situate their own research in relation to the general canon, and to identify studies that serve a bridging role, as we have done in the current study.
-
3. Which actors and institutions are creating knowledge in the field? Visualizing scientific landscapes can uncover relationships between authors, journals, institutions, and countries that generate conclusions about the actors shaping the field (Ralph and Arora Reference Ralph and Arora2024). Such analyses have been used, for example, to guide science policy and funding decisions (Ciarli and Ràfols Reference Ciarli and Ràfols2019).
Despite its advantages, literature mapping is not yet widely employed in political science. To facilitate its implementation, this study demonstrates the capabilities of literature mapping for studying complex concepts in political science, focusing on the illustrative example of scholarship on “organizing.” The methods used are documented in detail in the appendices, facilitating the replication and expansion of our approach.
Despite its advantages, literature mapping is not yet widely employed in political science. This study demonstrates the capabilities of literature mapping for studying complex concepts in political science, focusing on the illustrative example of scholarship on “organizing.”
ORGANIZING AS A CASE STUDY
We demonstrate the capabilities of literature mapping by using the term “organizing” in the context of political action. We focus on this term because there has been a recent surge in research on this topic (see appendix C), yet the meanings attributed to it have been so diverse that its meaning has ultimately become opaque (Han, McKenna, and Oyakawa Reference Han, McKenna and Oyakawa2021). Given fears of democratic erosion in advanced democracies, scholars have a renewed interest in organizing as a central pillar of democracy (Han, McKenna, and Oyakawa Reference Han, McKenna and Oyakawa2021; Woodly Reference Woodly2021).
A recent review of organizing focuses on the US context and differentiates the strategic logic that underlies organizing from other types of collective action (Han, Baggetta, and Oser Reference Han, Baggetta and Oser2024). Using a map of scholarship on organizing in the United States, the review identifies central areas in the literature. Our study adds to this work in three main ways. First, we provide a methodological description of the visualization and mapping method. Second, we expand the geographical scope of the mapping to include global scholarship on organizing beyond the US context. Third, we present maps and analyses of research on organizing and democracy that provide methodological guidance to researchers seeking to implement these techniques in their research.
To this end, we define the following three research questions in our illustrative analysis of organizing and democracy, which we answer through visualization and mapping techniques:
-
1. What thematic areas and sub-areas of scholarship exist globally that have studied organizing as a distinct concept? (RQ1, “Thematic areas”) This question also draws our attention to the relationship between these thematic areas in terms of their level of autonomy and interdependence.
-
2. Which of these areas includes substantial research on the connection between organizing and democracy? (RQ2, “Organizing and democracy”) Answering this question also clarifies which areas do not currently focus on this connection but could do so in the future.
-
3. What themes exist in the canonical literature on organizing, how do they relate to each other, and which key studies belong to each theme? (RQ3, “Canonical literature”) Answering this question helps position future research on organizing in relation to core theories.
Before proceeding to the case study analysis, we compare techniques that are commonly used to review and map a scientific field and then highlight the advantages of the literature mapping approach we used.
COMPARISON OF PREVALENT TECHNIQUES
Bibliometric methods employ searching, analyzing, and mapping techniques. For searching, many studies in political science use a basic search string comprising a few core terms determined by the authors (e.g., Boulianne, Oser, and Hoffmann Reference Boulianne, Oser and Hoffmann2023). However, bibliometric experts have found that basic searches, especially in emerging fields, may omit important related terms (Huang et al. Reference Huang, Schuehle, Porter and Youtie2015).
A central challenge in devising more sophisticated methods is achieving a balance between recall and precision (Huang et al. Reference Huang, Schuehle, Porter and Youtie2015). Each search term may retrieve false positives and false negatives. Although false positives can be eliminated manually, false negatives can be identified only by expanding the search. Bibliometric experts have devised semi-automatic techniques that balance recall and precision, using systematic transparent thresholds: these methods tolerate some levels of false negatives and positives while attempting to minimize both (Huang et al. Reference Huang, Schuehle, Porter and Youtie2015). We outline the adaptations of these methods that we implemented in the “Data and Methods” section. Although these adaptations resolve the problem of false positives, they may miss some relevant results. However, for versatile terms like “organizing,” these adaptations are necessary.
After defining the search and constructing the dataset, bibliometric analysis can be used to explore various types of relationships between publications, including citation, coauthorship, co-citation (two publications that cite the same source, see Small [Reference Small1973]), and co-occurrence (the appearance of two keywords in the same source; for a comprehensive review of types of relationships that can be analyzed, see van Eck and Waltman [Reference Van Eck, Waltman, Ding, Rousseau and Wolfram2014]). Although the choice of which relationship to analyze depends on the research objectives, choosing the methods and tools for the analysis involves additional considerations. For example, to identify themes within a research field, scholars have successfully used topic modeling techniques (Ambrosino et al. Reference Ambrosino, Cedrini, Davis, Fiori, Guerzoni and Nuccio2018). However, topic modeling does not analyze the relationship between themes and does not support the analysis of citation relationships. It also requires full-text access and additional visualization tools.
For scholars interested in visualizing relationships among publications, bibliometric mapping offers a useful solution (Van Eck and Waltman Reference Van Eck, Waltman, Ding, Rousseau and Wolfram2014). VOSviewer—which uses a unified approach for distance-based mapping and clustering called visualization of similarities (VOS; Waltman, van Eck, and Noyons Reference Waltman, van Eck and Noyons2010)—has several advantages over the alternatives. Compared to general statistical software, it creates more visually comprehensible maps by overcoming problems such as label overlap (Van Eck and Waltman Reference Van Eck and Waltman2010) and circular maps (Van Eck et al. Reference Van Eck, Waltman, Dekker and Van Den Berg2010). VOSviewer is also better suited than other bibliometric tools for both map creation and viewing, and it integrates files directly from major databases. It enables flexible viewing settings and an accessible interface that does not require any background in computer science or statistics.Footnote 1 In addition, unlike most statistical software packages, VOSviewer is free. Although the software is not open source, its features have made it the most popular software tool for bibliometric mapping purposes (Pan et al. Reference Pan, Yan, Cui and Hua2018). For a comparison of VOSviewer with other bibliometric mapping tools, see van Eck and Waltman (Reference Van Eck and Waltman2010; Reference Van Eck, Waltman, Ding, Rousseau and Wolfram2014), Pan et al. (Reference Pan, Yan, Cui and Hua2018), and McAllister, Lennertz, and Mojica (Reference McAllister, Lennertz and Mojica2022).
To determine the number of clusters in the map, VOS implements a resolution parameter. The larger the value of this parameter, the larger the number of clusters in the map. When this parameter is set to the default setting of 1.0, the clustering equation reduces to the well-known modularity function (Newman and Girvan Reference Newman and Girvan2004). However, modularity-based clustering may fail to identify small clusters, whereas the resolution parameter facilitates identifying clusters of varying sizes (Waltman, van Eck, and Noyons Reference Waltman, van Eck and Noyons2010, 631). Several studies have shown how adjusting the resolution parameter can yield useful scholarly insights (Fils and van Eck Reference Fils and Van Eck2018; Waltman, van Eck, and Noyons Reference Waltman, van Eck and Noyons2010).
In addition to these bibliometric tools, there has been a recent surge of artificial intelligence tools to assist scholars with literature reviews (Wagner, Lukyanenko, and Paré Reference Wagner, Lukyanenko and Paré2022). AI’s strengths in this domain include complex semantic meaning analysis using natural-language processing (NLP) methods and deep-learning capabilities that can potentially learn to replicate researchers’ decisions. However, available AI tools are still in the early development stages and do not yet offer a comprehensive solutions suite for literature mapping research.
Available AI tools are still in the early development stages and do not yet offer a comprehensive solutions suite for literature mapping research.
DATA AND METHODS
Implementing a research agenda using a visualization and mapping approach has three main stages as depicted in figure 1. In this section, we present the approach we took at each stage to answer our research questions on “organizing.” For a detailed guide on how to perform the steps presented in figure 1, see appendix D.
Step-by-Step Flowchart for Visualizing Scientific Landscapes
Search Strategy
Choosing the database: We sourced the data on scholarly works from the Web of Science (WoS), which has been identified as a leading scientific literature database with high accuracy and smooth integration of output files with visualization software (Visser, van Eck, and Waltman Reference Visser, van Eck and Waltman2021).
Developing the search term: The literature on bibliometric searches focuses on expanding a core dataset created by using search terms that retrieve mainly relevant results (Arora et al. Reference Arora, Porter, Youtie and Shapira2013; Chen and Song Reference Chen and Song2019; Huang et al. Reference Huang, Schuehle, Porter and Youtie2015). However, for complex terms with versatile meanings, like organizing, constructing this core dataset proves difficult. A simple search for “organizing” in the WoS “Topic” field yielded more than 80,000 mainly irrelevant results.Footnote 2 To meet our main challenge of focusing the search, we developed a strategy that combines existing approaches (Arora et al. Reference Arora, Porter, Youtie and Shapira2013; Huang et al. Reference Huang, Schuehle, Porter and Youtie2015), which we call a “targeted lexical search.” To narrow the search, we adapted techniques from two established bibliometric studies approaches: a core lexical search and an expanded lexical search (Huang et al. Reference Huang, Schuehle, Porter and Youtie2015).
In a core lexical search, researchers identify search terms through a literature review and subsequently vet them based on experts’ opinions to yield a core dataset (Arora et al. Reference Arora, Porter, Youtie and Shapira2013; Huang et al. Reference Huang, Schuehle, Porter and Youtie2015). In an expanded lexical search, researchers extract frequently appearing keywords from the core dataset. These keywords become candidate search terms and are vetted using a “noise ratio” measurement: an estimate of the percentage of irrelevant records retrieved by the candidate term based on a comparison with the core dataset (Arora et al. Reference Arora, Porter, Youtie and Shapira2013; Huang et al. Reference Huang, Schuehle, Porter and Youtie2015). To further enhance the precision of these strategies, some candidate terms are formatted as contingent terms, meaning that they are included in the search only when appearing alongside another term.
In this study, rather than using “organizing” as an independent search term, we identified contingent terms that appeared next to it and that modified its meaning (e.g., community organizing) through a literature review. We made a list of 29 candidate terms that met this contingency format. We then developed a modified version of the “noise ratio” measurement. In our case, because the benchmark core dataset did not exist, we created an equivalent measurement, termed the “hit ratio,” and calculated it for each candidate term. The hit ratio is the proportion of relevant results of the 10 most-cited records retrieved when searching for the candidate term. Following the expanded lexical search (Huang et al. Reference Huang, Schuehle, Porter and Youtie2015), we applied a 70% threshold to determine whether to include or exclude each contingent term. That is, for each candidate term, if 7 or more of the 10 most-cited results were relevant, the term was included. We manually determined the relevancy of results based on core definitions from the literature (appendix H). This process yielded 21 contingent terms that we used in our final search (for a full list of terms, hit ratios, and the final Boolean search term, see appendix A). This strategy yielded a dataset of 2,334 records on the WoS.Footnote 3
Screening the dataset: To validate the method, we manually vetted all search results in the dataset. After two independent coders determined whether a record was relevant (see appendix B: Vetting Instructions), the vetted dataset included 2,156 records, comprising 92.4% of the original dataset. Intercoder reliability was 95.7%. This article’s tables and figures all use the vetted dataset. The high rate of relevant results demonstrates our search strategy’s strength, producing a low noise ratio of only 7.6%. This search strategy’s high hit rate suggests that this method may be used without manual vetting. See appendix C for a detailed characterization of the resulting dataset and appendix D for more information on search strategy development.
Map Creation
Choose visualization tool: We used VOSviewer (version 1.6.20) to construct and visualize bibliometric networks (Van Eck and Waltman Reference Van Eck and Waltman2010). Appendix D includes a basic walkthrough for using the software and references of additional guides and tutorials.
Choose types of maps: We chose to create a co-occurrence map, which shows relationships between terms that appear frequently in the Author Keywords and Keywords Plus (a list of frequent words extracted by WoS’s algorithm) fields on WoS. This map shows all the frequent keywords in the dataset whether they were part of the original search or not, thereby providing a comprehensive representation of the themes in the field. We also created a co-citation map that shows relationships among frequently cited references in the dataset. These two types of maps were best suited to answer our research questions, as shown in the “Results” section.
Prepare supplementary files: Following best practices in the literature (McAllister, Lennertz, and Mojica Reference McAllister, Lennertz and Mojica2022), we created thesaurus files for the maps, as documented in appendix E.
Choose map preferences and create maps: Map preferences can be adjusted to accommodate different research objectives. We relied on common practice in the literature and used the preferences of full counting, a threshold of 15 occurrences or citations, and the default clustering resolution parameter of 1.0 (Van Eck and Waltman Reference Van Eck, Waltman, Ding, Rousseau and Wolfram2014). More information on setting map preferences is included in appendix D. Data and replication files are available in Shoshan and Oser (Reference Shoshan and Oser2024).
Map Analysis
Label clusters: We labeled the clusters in the maps manually. In the co-occurrence map (figure 2), we determined the labeling after observing the list of keywords associated with each cluster. For the co-citation map (figure 4), we determined the labeling based on a review of the titles and abstracts of all references associated with each cluster.
Main Topics of Thematic Interest: Terms Co-occurrence Map
n = 85 out of N = 4,893.
Leverage mapping tools to obtain relevant insights: We used VOSviewer’s features to draw insights about the literature that go beyond the thematic labeling of subfields. Specifically, for the two maps in this study, we leveraged VOSviewer’s distance-based mapping to assess the relationship between subfields, distinguishing tightly connected subfields from disconnected subfields. This analysis allowed us to identify potential gaps in the literature when analyzing the co-occurrence map (figure 2). Leveraging the software’s viewing options to focus on one keyword—democracy—that we considered important, we visualized its links within the general map and identified disconnected subfields where more research on this topic could yield new insights. In the co-citation map (figure 4), we identified clusters of canonical literature and their relationships to guide scholars interested in pursuing future research on organizing.
RESULTS
Thematic Analysis
To answer RQ1 (“Thematic areas”), we created a co-occurrence map (figure 2) that visually represents the co-occurrence of keywords in the dataset. It shows the topics and subfields found in the study of organizing and their relationships. Appendix F includes the list of keywords and their frequencies.
In the map, the size of the nodes represents the frequency of the keywords. The links between nodes represent the co-occurrence of terms across records. Terms with similar relationships to other terms are marked by their color, map location, and the thickness of connecting lines. Two terms appearing close to each other are therefore more strongly connected than those appearing at a greater distance.
Figure 2 shows that scholarship on organizing is structured around five distinct clusters of thematic interests. Observing the most frequent terms, we characterized each of these clusters’ main themes, from largest to smallest: (1) Labor (red), (2) Community organizing (green), (3) Race/gender (yellow), (4) Social movements (blue), and (5) Urban studies and community development (purple).
An analysis of the terms’ locations shows that the labor and community organizing subfields, although connected to other themes, are relatively autonomous. The absence of red or green nodes within other clusters suggests that research on these themes tends to be more independent than research on social movements, race/gender, and urban studies.
An examination of the terms’ location within the labor and community organizing clusters also reveals the existence of sub-areas within these themes. In the labor cluster, peripheral terms included all keywords related to unions, indicating that there is a body of literature on union organizing that is relatively disconnected from community organizing and social movements. Likewise, in the community organizing cluster, peripheral terms include keywords such as intervention and prevention, indicating the existence of a body of literature in applied research fields (e.g., healthcare, psychology) on intervention and prevention programs that have a community organizing component.
Organizing and Democracy
Based on our thematic analysis, we leveraged VOSviewer’s viewing options to answer RQ2 (“Organizing and democracy”). To this end, we visualized the relationships between the keyword “democracy” and other keywords in the map, as highlighted in figure 3.
Democracy in the Terms Co-Occurrence Map
This map, structurally identical to figure 2, visually highlights the keyword “democracy” and its relationship with other keywords.
Figure 3 shows that the relationship between organizing and democracy is studied across a wide range of fields, as evidenced in the links between “democracy” to keywords in all five clusters. Despite this interdisciplinary reach, some sub-areas were significantly detached from the study of democracy. Particularly noticeable peripheral areas in figure 3 are those in the labor cluster focusing on unions, in the community organizing cluster focusing on prevention and intervention programs, and in the social movements cluster focusing on social media, none of which have a relationship to the keyword “democracy.”
We used these observations to characterize potential gaps in the literature. Manually reviewing records in the dataset related to the study of unions, we found that the few studies explicitly exploring the relationship between unions and democracy focused on unions’ internal democratic practices without relating to unions’ contribution to democracy on the societal level. Similarly, in health-oriented research, experimental research on the impact of community organizing interventions has focused on measuring behavioral outcomes related to health, and not on organizing’s impact on leadership capacity or on organizational or political structures. Likewise, research on social media and organizing has focused primarily on the influence of social media on social movements. But this literature has not directly addressed the question of how organizing in the age of social media affects collective democratic outcomes or the representation of marginalized groups. We conclude that in these three sub-areas the connection between organizing and democracy has been understudied and future research on this connection can yield new and interesting insights.
Canonical Literature
Scholars producing research on the connection between organizing and democracy need to orient their research in relation to the canonical literature of the field. We facilitate this orientation by answering RQ3 (“Canonical literature”): we identified the body of literature comprising the field’s canon, structure, and relationship to the study of democracy by analyzing the co-citation map (figure 4).
Canonical Literature: Co-citation Map
n = 89 out of N > 70,000 references.
Figure 4 documents the body of literature frequently cited by scholars of organizing. The lines between nodes represent the co-citation of two references by the same record. The map contains five clusters, representing five areas of canonical literature. Observing the titles and abstracts of the references in each cluster, we characterized the thematic content of each area, beginning with the largest cluster: (1) Community organizing (green), (2) Labor (red), (3) Civic associations (gray), (4) Social movements (blue), and (5) American democracy (orange). For a detailed thematic characterization of each cluster, see appendix I; for the full list of references ordered by thematic clusters, see appendix G.
The canonical literature clusters identified in the co-citation map (figure 4) clarify the theories and concepts that underlie the research themes in the keywords analysis (figure 2). Specifically, we observed that the social movements and community organizing clusters in figure 2 draw on more than one group of theories. Analyzing the main content of the classic studies in each cluster of the co-citation map (see appendix I) and looking at the location of the canonical literature clusters in the co-citation map, we concluded that the social movements area in figure 2 draws on classic social movement studies (the blue cluster in figure 4) and on classic American democracy studies (the orange cluster in figure 4). Likewise, the community organizing area in figure 2 draws on classical studies of community organizing (the green cluster in figure 4), as well as on classic American democracy studies (the orange cluster in figure 4) and classic civic associations studies (the gray cluster in figure 4).
This analysis of the canon helps scholars pursuing research on organizing, in general, and on its connection to democracy, in particular, to navigate the field. For example, to pursue research on the understudied connection between unions and collective democratic structures and norms, scholars would benefit by drawing on the set of classic studies included in the canonical labor cluster and the American democracy cluster. In bridging these two classic bodies of literature, scholars may identify relevant concepts and theories in the canonical social movement cluster that, as its location in the map shows, has traditionally served a bridging role between these two research areas.
We further observed that the research areas of gender, race, and urban development that appear in the terms co-occurrence map (figure 2) are absent in the co-citation map (figure 4). We hypothesize that this difference is due to the temporal evolution of the field, in which newer areas of research, such as race and gender, have not yet consolidated into a clearly demarcated canon. We offer this hypothesis as a future research direction that can be addressed using a temporal literature mapping analysis that goes beyond the scope of our study.
Conclusion
Our study demonstrates how visualizing scientific landscapes can contribute new levels of analysis for political science researchers aiming to enhance their understanding of the literature on complex topics. Using a terms co-occurrence map, we identified sub-areas of research where the connection between organizing and democracy has been understudied and so potentially merit additional research. Next, using a co-citation map of highly cited references, we helped guide scholars who wish to address these gaps by identifying the specific classic studies from which their research can benefit, as well as groups of studies that can help them synthesize research areas.
The mapping of scholarship on organizing presented here serves as an illustration of this method’s capabilities, but it is not exhaustive. Future directions for research that stem from this study include a temporal analysis to identify emerging themes and a thematic analysis to identify sub-areas of research that merit a meta-analysis. For example, using the terms co-occurrence map (figure 2), we identified a significant body of literature that merits a meta-analysis on the effects of health-related intervention programs with a community organizing component. Literature mapping can also facilitate the identification of citation bias in meta-analyses, as has been done in other disciplines (e.g., Bellos Reference Bellos2021). Taken together, our illustrative analysis shows the multiple ways in which literature mapping can generate valuable insights for researchers who aim to produce comprehensive data-driven reviews of political science research fields.
SUPPLEMENTARY MATERIAL
To view supplementary material for this article, please visit http://doi.org/10.1017/S1049096524001057.
ACKNOWLEDGEMENTS
We thank Hahrie Han and Matthew Baggetta for their expert contributions to refining the literature search terms and citation mapping output. We also thank Omri Cohen for his valuable research assistance. Previous versions of this article were presented at the Midwest Political Science Association conference in April 2023 and the Southern Political Science Association Conference in June 2023. We thank all participants for their insightful comments and are especially grateful to Anthony Chergosky and Karen O’Connor for suggestions that strengthened this article. It developed in the course of producing a citation map for an article published in the Annual Review of Political Science: Hahrie Han Matthew Baggetta, and Jennifer Oser, Reference Han, Baggetta and Oser2024, “Organizing and Democracy: Understanding the Possibilities for Transformative Collective Action,” Annual Review of Political Science 27, https://doi.org/10.1146/annurev-polisci-041322-043040.
FUNDING STATEMENT
Research for this article was supported by funding from the Israel Science Foundation Individual Research Grant to J.O., grant number 1246/20, and from a European Union grant to J.O., European Research Council, project number 101077659.
DATA AVAILABILITY STATEMENT
Research documentation and data that support the findings of this study are openly available at the Harvard Dataverse at https://doi.org/10.7910/DVN/FGBBSO.
CONFLICTS OF INTEREST
The authors declare no ethical issues or conflicts of interest in this research.
Open access
