To save content items to your account,
please confirm that you agree to abide by our usage policies.
If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account.
Find out more about saving content to .
To save content items to your Kindle, first ensure no-reply@cambridge.org
is added to your Approved Personal Document E-mail List under your Personal Document Settings
on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part
of your Kindle email address below.
Find out more about saving to your Kindle.
Note you can select to save to either the @free.kindle.com or @kindle.com variations.
‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi.
‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.
Research on informal digital learning of English (IDLE) has grown substantially, yet a systematic analysis of its methodological foundations is lacking. To map current practices, this methodological review examines 104 peer-reviewed studies (2015–2025) indexed in the Arts & Humanities Citation Index or the Social Sciences Citation Index. The findings reveal a dominant pattern of methodological homogeneity, characterized by heavy reliance on non-experimental designs (94.23%), self-report instruments (notably questionnaires; 69.49% of quantitative tools), and descriptive and correlational analyses (80.44% of quantitative analysis strategies). This profile is understandable, as the informal, nonlinear, and learner-driven nature of IDLE aligns naturally with exploratory approaches. However, this focus may limit the investigation of causal mechanisms and theoretical development. To support the field’s evolution, we propose a strategic diversification of methods. Integrating longitudinal designs to trace development, complementing self-reports with ethically collected and activity-sensitive behavioral and trace data, and leveraging new analytical tools such as learning analytics and, where appropriate, AI-assisted analysis can enrich inquiry and help build more robust, explanatory models of informal digital learning.
Can interviews or a focus group improve the causal inferences drawn from experiments? Can quantitative text analysis help develop workflows as a qualitative scholar? Can we learn from a single case in a way that helps us with a statistical model? There is much to learn from the careful use of all these methodological combinations. The Practice of Multi-Method Research is aimed at practical researchers: from undergraduates preparing for an honors thesis, to graduate students designing a dissertation, through to seasoned scholars considering a new approach for their next set of studies. It offers a hands-on, practical guide to combining research across various methodological traditions: qualitative, machine learning, and quantitative approaches to concepts and measurement, adding quantitative and data-science components to process-tracing designs and to qualitative case studies in general, how qualitative research can strengthen regression-type designs, and how to mix qualitative elements with experiments..
How can regression-type studies, which try to get causal inferences by controlling for variables until whatever is left is plausibly a causal relationship, benefit as much as possible from qualitative research? This chapter offers three qualitative research designs that can help test and improve quantitative causal inferences based on control variables. First, we consider process-tracing designs that seek to discover previously unknown confounding variables that can be added as new control variables, thereby refining and improving the statistical model. Second, we discuss process tracing that seeks to confirm that control variables have a direct connection to the outcome variable, and therefore are relevant controls. Third, we discuss process tracing to check whether control variables might be posttreatment and therefore at least partially harmful. Finally, we provide a demonstration of the kind of iterated interaction that we have in mind between statistical efforts at causal inference and qualitative research to learn more about the assumptions behind the model, involving the relationship between labor union strength and the prevalence of employment law across U.S. states.
How can mixed-method research add value to experiments? There is much to learn by adding qualitative components to experiments that can test for problems related to measurement; networks and spillover among experimental subjects; issues related to the meaning of treatments, outcomes, and other aspects of the experiment for participants; the exhaustiveness of the intended causal content of the treatment; and moderation effects related to subgroups of participants, or to differences across implementations of an experiment in divergent contexts. Some of these areas relate to the goal of theoretical generalization from the initial experiment, but most of them also have connections with the meaning and quality of the causal inference for the experiment at hand.
Process tracing methods, which involve ways of using information within a single case to think carefully about questions of social-science theory, have become one of the leading qualitative methods in political science and related fields. Can mixed-method approach to process tracing work without destroying the qualitative logic of the method? This chapter argues that two distinct forms of process tracing, involving testing of existing causal theories and discovery of new theories, each have room for mixed-method strategies. We show that qualitative components can become steps within a testing-based process tracing argument, and offer guidelines for structuring such studies. We then offer a novel approach to combining machine-learning techniques like LASSO regression with discovery-based process tracing, illustrating the design with an application to the rise in visibility of U.S. far right broadcaster Alex Jones.
How can quantitative methods support case studies in general? One common task in case-study research is presenting audiences with a large collection of texts. Text-as-data methods may facilitate this task by summarizing the overall collection more quickly and understandably than we can usually do ourselves, and by locating individual documents within the collection. We offer an application, showing how text-as-data methods effectively summarize the large collection of legal documents available regarding defendants in cases related to the January 6th invasion and facilitate case-study exploration and understanding of specific individuals. We also discuss how mixed-method approaches may save time and resources as a way of moving across levels of analysis within a case-study design. We wind up the chapter by raising a note of caution regarding common attempts to use statistics as a test of generalizability for case studies and regarding the somewhat less common but still recurrent designs that mix game theory and qualitative research. Both of these approaches can work, and can offer insights, but both are harder than they may initially appear.
Research designs can combine qualitative, quantitative, experimental, and data-science methods through careful planning, attention to details and assumptions, and methodological execution. This chapter introduces the book's perspective, arguing that designs that combine methods carefully and with close attention to their details will outperform designs where two or more methods are carried out in parallel and then compared. The chapter also considers a range of practical and philosophical questions related to whether it makes sense to combine methods. Is one method always best, such that it isn't worth trying? Do methods speak fundamentally different languages, such that there is no point? Are methods and problems inextricably linked, such that the question is irrelevant? We argue against each of these views.
This chapter builds a theory of local community protection from populist state violence, focusing on the “meso” level where state agents and communities interact. After describing the characteristics of this new type of political violence, it explains how two key variables – the moral commitment of local elites (Catholic priests and other religious actors) and the capacity of local social organizations (in this case, the Catholic parish) – facilitate community protection. Commitment and capacity have a path-dependent character, reflecting formative experiences for clergy, exposure to Catholic religious orders with histories of public action, and long church embeddedness in local communities. These two independent variables operate via six mechanisms: disrupting operations and reducing targets function through interactions with local state agents, threatening accountability and reintegrating victims involve partnerships with civil society, and raising moral salience and building solidarity work at the level of the local public.
The integration of generative AI into humanities research presents both new opportunities and significant methodological challenges. While generative AI enables more sophisticated textual, visual, and historical analysis than ever before, it also introduces novel methodological risks. Prompts now function as measurement instruments, construct validity is entangled with model fluency, the increasing abstraction and subjectivity of tasks challenge single-answer evaluations, small changes in prompt or model configuration can materially alter substantive conclusions, and model training encodes strong theoretical assumptions prior to human interaction. This article advances a theory-first framework for generative-AI-assisted humanities research that traces how theoretical commitments intervene across the research lifecycle. Drawing on examples from the computational humanities and social sciences, it argues that theory operates as methodological infrastructure that constrains degrees of freedom, supports reproducibility, and enables principled iteration. By reframing generative AI as a site of theoretical inscription, the article outlines practical strategies for building robust and theoretically grounded humanities research in the era of generative AI.
This chapter describes the study’s research design. Protest event data from the three contexts over three decades, gathered with an intensive data collection strategy, was aggregated into a dataset of forty large far-right demonstration campaigns. The chapter describes the application of qualitative comparative analysis (QCA) and process-tracing techniques to detect patterns of causal conditions and then to evaluate rigorously the processual causation underlying those patterns. In social movement research and scholarship on the far right, single-case studies abound, providing rich qualitative depth but limited generalisability. At the same time, large-N quantitative research often lacks that depth of information that enables robust causal inferences. The main thrust of the research design applied here is overcoming these deficiencies.
Chapter 1 introduces the study’s core puzzle and overall logic of inquiry. It discusses main themes, locates arguments relative to relevant scholarship, and establishes the analytical framework. Early in the chapter, the puzzle of varying illiberal electoral outcomes is presented and contextualized. Captured by two distinct yet related indicators – illiberal voting and post-neoliberal populist magnitude – illiberal electoral outcomes not only varied persistently across countries but also signaled the high salience of economic issues in postcommunist Europe. The next section establishes the rationale for explaining outcomes by drawing insights from Latin America – another semi-peripheral space that experienced consequential neoliberal junctures. Having argued, based on key economic and political parallels between the two regions, that a critical juncture approach is appropriate also for making sense of developments in Eastern Europe, I spell out the work’s central propositions and highlight theoretical, methodological, and empirical contributions. The final sections discuss matters of research design and evidence – namely, the mixed method approach, case studies, and quantitative and qualitative data, including 100 interviews – as well as the book’s organization.
All linguistic research has the potential to reproduce or challenge racial notions.
—Linguistic Society of America Statement on Race (2019)
The LSA Statement on Race stems from a larger conversation around undertheorized treatment of race and ethnicity in linguistics research and practice. In this commentary, we define racial identity and ethnicity and explain their relevance for linguistic research. We discuss considerations that linguistic researchers should take prior to research, during study design, and following research, and we offer specific recommendations when soliciting or using race and ethnicity data. These recommendations aim to help researchers avoid social harm, ensure ethical compliance and research integrity, and improve descriptive accuracy, especially for undersampled groups, by balancing research transparency with generalizability. We consider issues germane to collecting self-disclosures of ethnicity and racial identity in a range of study types spanning several subfields of linguistics. We give concrete examples of questions that may arise in planning studies in computational and corpus-based linguistics, formal linguistics, experimental linguistics, and qualitative linguistics. We speak to ethical considerations, including the importance of using locally constructed labels, analyst positionality, and respect for communities. Our goals are to provide linguistic researchers with a firmer basis for conceptualizing racial identity and ethnicity particularly as pertains to linguistics, and to supply a guide that aids linguists in reflecting on their own study design, positionality, and responsibility to participants and communities.
Quality data are necessary to test research questions about psychopathy. This chapter describes the research design of the Incarcerated Serious and Violent Young Offender Study (ISVYOS). The ISVYOS was initiated in 1998 and has been ongoing ever since. In this chapter I describe the ISVYOS that was used in the book, procedures for interviewing youth and collecting data in adulthood, control variables that were accounted for in the different analyses in this book, and the different measures of psychopathy that were available. This book focuses on 535 incarcerated youth who received an interview and rating on the PCL:YV. Many of these youth received psychopathy assessments using additional measures. To give readers a better sense of how psychopathy manifests in youth, I describe procedures for rating the PCL:YV and discuss and interpret different responses that youth provided that were used as indications of the presence of specific psychopathy traits. This includes describing manifestations of each item from the Interpersonal, Affective, and Lifestyle factors from the PCL:YV.
Research syntheses have demonstrated that pronunciation instruction works, which means that whether instruction is effective is no longer an open question. Instead, contemporary intervention research has shifted to investigating how instruction can be further optimized, asking targeted questions about the instructional features that catalyze learning. In this paper, I examine the concept of instructional optimization, focusing on anticipated effect sizes (gains). I outline a four-pronged empirical approach to provide robust data for designing optimal pronunciation interventions. First, I describe the need for replication studies, which provide insight into the precision and stability of effects across distinct research samples and contexts. Second, I advocate for a systematic approach to study design. In such an approach, which is closely tied to the principles of replication, one or two variables are manipulated at a time, leading to a set of maximally comparable studies that lend insight into the impact of specific variables. Third, I explain the need to situate instruction within a longitudinal perspective to examine how robust and durable instructional gains are. Finally, I turn to adaptive approaches, where the surface format that instruction takes is highly variable and responsive to learner needs while the adaptive decision tree that generates the form is fixed and replicable.
Clinical and Translational Science Awards (CTSAs) are positioned to enhance the integration of rigorous implementation research methods into projects across their networks, but lack a systematic, standardized process to do so. This study introduces the Dissemination and Implementation Research Capability Self Survey (DIRC-SS), a pragmatic instrument to evaluate and integrate implementation science methods in traditional research activities.
Methods:
We developed the 15-item DIRC-SS to assess researchers’ use of implementation research methods across five key constructs. Its reliability (inter-rater agreement and internal consistency) and sensitivity (change over time) were examined in 10 NIH-funded research projects via ratings assigned by the research teams and by implementation science experts at baseline and one year later.
Results:
The DIRC-SS total score demonstrated good internal consistency and inter-rater reliability increased over one year. Although the research team ratings did not change significantly over time, the expert ratings significantly increased, and effect sizes across research teams and expert raters were large in this small sample study.
Conclusions:
The DIRC-SS demonstrated good internal consistency reliability and moderate inter-rater reliability. It effectively distinguished between different levels of implementation research methods integration. Unlike tools focused on grant proposals or final reports, the DIRC-SS can be used at any point in the research process by a research team as a self-survey, by implementation science experts in a consultation process, or across a CTSA program to characterize the implementation science methods employed across projects and highlight targeted areas for researcher education and training.
This chapter outlines the empirical strategy for studying policy triage, which occurs when limited administrative resources and growing policy stocks force agencies to prioritize certain implementation tasks over others. To measure policy triage, the analysis distinguishes between triage frequency and intensity. These dimensions together provide a nuanced assessment of overall implementation performance. The chapter also details the theoretical predictors of policy triage: whether central policymakers can shift blame for failures, whether implementing agencies can mobilize external resources, and whether they are internally committed to achieving policy goals despite resource constraints. To test these claims, the research design focuses on two policy areas — environmental and social policy — across six countries representing diverse administrative traditions. Data collection involves secondary document analysis and 157 expert interviews with implementation officials. By systematically capturing both formal and informal organizational practices, this methodology reveals the complex trade-offs inherent in modern public administration and underscores how different political and organizational conditions jointly shape policy triage.
Empirical research papers are a mix of technical skill and unwritten insider information. Providing practical guidance on how to design, analyze, and write about research, this engaging second edition is fully updated with expanded coverage of finding and using data, a topical running example, and new appendices introducing quantitative analysis techniques. It covers everything from crafting a question, theory, and hypotheses to choosing a research design, acquiring and analyzing data, drafting, peer review, and presenting your work. Practical strategies are combined with a step-by-step breakdown of every stage of the research design and writing processes, conveyed with clarity and humor. The intuitive presentation illustrates the core insights and concepts in a lively and accessible manner for readers, including those with no mathematical background and from fields beyond political science. New 'Common Challenges' boxes join a wealth of inspiring pedagogical features. Online resources include a revised Instructor's Manual, exercises and essays.
Elite donors are a crucial and sought-after source of funding for many nonprofit organisations, but there is a dearth of substantive empirical studies presenting primary data on such donors’ motivations, experiences and perspectives. There are challenges for social scientists in conducting interviews with elites, notably: gaining access to elite donors; developing sufficient rapport to discuss a topic that involves money and morals; and making sense of data without being dazzled by striking surface differences between elites and non-elites. These barriers have resulted in a long-standing over-reliance on secondary sources and on interviews with proxies such as foundation staff and wealth advisers. This paper reviews the small body of work that presents findings from interviews with elite donors and draws on my experience of conducting interviews with 46 wealthy UK donors, in order to critically analyse the implementation of this research design. This paper adds to the literature by extending understanding of elite donors’ reasons for agreeing to be interviewed and contributes to advancing third sector research by highlighting strategies to overcome challenges in conducting elite interviews in order to gain a less mediated understanding of the contexts, cultures and subjectivities of their focus of study.
This article provides a reassessment of the literature on the transformative impact of the EU on Turkey through the lens of the ‘Europeanisation research programme’. It relies on systematic examination of a sample of the literature based on substantive findings, research design and methods. It suggests that this sample displays limitations characteristic of the Europeanisation research programme and proposes to remedy these limitations by applying the research design and methods used therein for generating empirically based comparative research on Turkey.