Farm animal behaviour and welfare research may have limited generalisability if conducted on groups of animals characterised by a restricted set of individual factors and living conditions. In this sense, due to the contextual idiosyncrasies of individual testing sites, the results of individual studies could only be valid for the particular group of animals and thus may not always be reproducible. However, robust results are necessary to ensure that outcomes are broadly relevant, as this is vital if such results are intended to contribute to improved husbandry and management conditions, and ultimately, better animal welfare policies. So-called Big Team Science approaches (hereafter BTS), which pool resources across large numbers of researchers to facilitate data collection from larger and more diverse samples, can offer a resource-efficient opportunity to tackle this problem and increase the external validity of scientific results. Here, we discuss the benefits and challenges associated with BTS approaches in farm animal behaviour and welfare research. As contextual factors vary considerably across species kept for different products and in diverse husbandry settings, we conducted a snapshot review focusing on one of these species, the domestic goat (Capra hircus). Here, we found that the majority of studies were conducted at single sites and on only one breed. Finally, we introduce the ManyGoats initiative, which seeks to increase the generalisability of findings in research on goat behaviour and welfare by implementing identical experimental protocols and simultaneously testing animals, including multiple breeds, across different facilities around the world.

Evaluation of reasoning language models gained importance after it was observed that they can combine their existing capabilities into novel traces of intermediate steps before task completion and that the traces can sometimes help them to generalize better than past models. We show that better performance results not only from test time algorithmic improvements or model sizes but also from letting tasks from impactful benchmarks inspire curricula for learning. We call this benchmark-driven selection of AI and show its effects on DeepSeek-R1, the first open-weight large reasoning model, using a novel sequential decision-making problem that we contributed to the philosophy category of Humanity’s Last Exam, a frontier academic AI benchmark. Steering development of AI by impactful benchmarks trades evaluation for learning and makes novelty of test tasks key for measuring generalization capabilities of reasoning models. Consequently, some benchmarks inspire curricula for post-training that improve external validity of the models. Understanding public benchmarks as mere test sets risks confusing unseen tasks measuring uncontaminated generalization with known tasks that improve external validity. This paper explores new avenues for experimentation in epistemology and shows that formal AI evaluation problems are helpful for understanding model capabilities and their evolution.

Real-world evidence (RWE) is reshaping how health technology assessment (HTA) bodies evaluate therapies, yet a critical gap persists: while transportability methods enable researchers to adapt treatment effect estimates from RWE studies in another jurisdiction to local contexts, the field remains unclear on when these analyses impact decision-making. This article synthesizes insights from HTA bodies, health technology developers (HTDs), and methodologists to clarify what is missing from today’s transportability conversation. We find that methodological progress has outpaced guidance on its utility; HTDs risk investing in transportability analyses without clear evidence that they reduce key uncertainties or affect recommendations. We identify three fundamental gaps: the absence of objective thresholds for determining whether transportability assumptions are adequately met; limited guidance on when quantitative transportability is needed versus qualitative assessment; and variation in HTA bodies’ evaluation and application of transported evidence. These gaps create a critical impasse where HTDs must decide whether to invest in such analyses, while HTA bodies lack standardized frameworks for their assessment. We argue that transportability cannot transition from academic exercise to a decision-making tool without explicit early dialogue between stakeholders, transparent reporting standards, and capacity building within HTA bodies. We present actionable recommendations to build HTA expertise, create a case study repository, and define contexts where transported RWE demonstrably impacts patient access. As evidence dossiers increasingly rely on nonlocal data, addressing these strategic and operational gaps becomes essential to help patients.

Internal and external validity are key criteria for evaluating the quality and usefulness of research on second language acquisition (SLA). Although considerable attention is given to internal validity, external validity—the extent to which findings can be generalized beyond the specific study conditions—may be neglected during peer review and subsequent interpretations of findings. This editorial argues that a lack of attention paid to external validity reduces the interpretability and responsible use of SLA research through overgeneralization and misinterpretation of results. Current initiatives aimed at improving generalizability, including replication, meta-analysis, and multisite studies, are discussed, and a framework for evaluating external validity is proposed. The editorial concludes by calling for more transparent reporting practices and more cautious interpretation of research findings, with the aim of promoting accurate interpretation, replication, and theoretically motivated follow-up research in SLA.

Can survey experiments replicate real-world behavioral intent and behavior? I study a population in rural Bangladesh (N ∼ 1600) along the banks of the Jamuna River, at risk of riverbank erosion and flooding. I compare their responses to questions about hypothetical movement behavior in vignette-experimental natural disaster scenarios (pre-monsoon, May–June 2021) with their migration intentions and actual migration 2–6 months later, following quasi-experimental real-world exposure. My results show that hypothetical as well as actual affectedness and risk shape migration intent and behavior in structurally similar ways, indicating sign-generalization over both treatments and outcomes. However, the vignette experiment approximates actual behavioral intent more closely than behavior, suggesting that real-world intention–behavior gaps can complicate external validity. Given a slim evidence base for generalizability over treatments and outcomes, this study contributes a crucial comparison from a rarely studied developing-country context on what we can learn from survey experiments.

This guest editorial highlights the importance of external validity in randomised clinical trials in psychiatry, using patients with borderline personality disorder as an example. We suggest strategies to improve the external validity of trials, including adjusting randomisation procedures and implementing detailed screening logs at the referral stage.

The dictator game has become a celebrated workhorse of experimental economics and social psychology. In the standard version of the game an individual is given a sum of money and must choose how to split this money between themselves and some other individual. In a variant of the game the individual must split the money between themselves and a charitable cause. This charity version of the dictator game has now been used in well over fifty studies and has provided critical insight on the motives behind giving. It also provides a simple tool that policy makers and practitioners can use to test the effect of interventions. In this paper we explain the different ways in which charity dictator games can and have been used. We also look at the external validity of charity dictator games and discuss the research questions that can be appropriately studied using them.

Within-study comparisons (WSCs) compare quasi-experimental results to an experimental benchmark and assess the extent to which quasi-experiments (QEs) sacrifice internal validity for external validity. WSCs inform decisions about when to use experiments or QEs, as well as methodological decisions (e.g., covariate selection) about how to implement QEs when they are necessary or preferred. We review the methodological literature on WSCs and present the findings from a WSC of a school voucher program as an example of the use of WSCs to inform policy debates involving the third sector. Prior literature and our results suggest that QE evaluations of third-sector institutions can produce low levels of bias, but only if analytic models include key proxy variables for voluntary self-selection into programs.

Online panels have become an important resource for research in political science, but the compensation offered to panelists incentivizes them to become “survey professionals,” raising concerns about data quality. We provide evidence on survey professionalism exploring three US samples of subjects who donated their browsing data, recruited via Lucid, YouGov, and Facebook (total $n = 3,886$). Survey professionalism is common, but varies across samples: by our most conservative estimate, we find 1.7% of respondents on Facebook, 7.$\color {black}6$% on YouGov, and 34$\color {black}.7$% on Lucid to be professionals (under the assumption that professionals are as likely as non-professionals to donate data after conditioning on observable demographics available from all online survey takers). However, evidence that professionals lower data quality is limited: they do not systematically differ demographically or politically from non-professionals and do not exhibit more response instability. They are, however, somewhat more likely to speed, straightline, and attempt to take questionnaires repeatedly. To address potential selection issues in donating of browsing data, we present sensitivity analyses with lower bounds for survey professionalism. While concerns about professionalism are warranted, we conclude that survey professionals do not, by and large, distort inferences of research based on online panels.

Prior research shows that, when making causal inferences, people can control for alternative causes. However, these studies utilize artificial inter-trial intervals on the order of seconds; in real-life situations, people often experience data over days and weeks (e.g., learning the effectiveness of two new medications over multiple weeks). Across two experiments, participants learned about two possible causes from data presented either in a more naturalistic paradigm (one trial per day for multiple weeks via smartphone) or in a traditional trial-by-trial paradigm (a rapid series of trials). The results show that people can control for alternative causes when learning over long timeframes, but they also exhibit non-normative discounting. The results also reveal that the extent to which people control and learn simple relations is suboptimal across both short and long timeframes.

Intergroup attitudes and identity ties can shape foreign policy preferences. Anti-Muslim bias is particularly salient in the USA and the UK, but little work assesses whether this bias generalizes to other countries. We evaluate the extent of anti-Muslim bias in foreign policy attitudes through harmonized survey experiments in thirteen European countries (N=19,673). Experimental vignettes present factual reports of religious persecution by China, counter-stereotypically depicting Muslims as victims. We find evidence of anti-Muslim bias. Participants are less opposed to persecution and less likely to support intervention when Muslims, as opposed to other religious groups, are persecuted. However, this bias is not present in all countries. Exploratory analyses underscore that pre-existing intergroup attitudes and shared group identity moderate how group-based evaluations shape foreign policy attitudes. We provide extensive cross-national evidence that anti-Muslim bias is country-specific and that social identity ties and intergroup attitudes influence foreign policy preferences.

Three effects of apparently superficial changes in presentation (“framing effects” in abroad sense), were replicated together in the same repeated linear public goods experiment with real financial incentives. First, 32 repetitions were presented as four phases of 8 repetitions with a break and results summary in between. Contribution levels decayed during each phase but then persistently returned to about 50% after each re-start. Second, subjects contributed more when the payoff function was decomposed in terms of a gift which is multiplied and distributed to the other players, rather than the equivalent public good from which everyone benefits. Third, subjects contributed more following a comprehension task which asks them to calculate the benefits to the group of various actions (the “We” frame), rather than the benefits to themselves (the “I” frame). These results suggest that aspects of presentation may have strong and replicable effects on experimental findings, even when care is taken to make the language and presentation of instructions as neutral as possible. Experimental economists should therefore give careful consideration to potential framing effects—or, better still, explicitly test for them—before making claims about the external validity of results.

Accountability—the expectation on the side of the decision maker that she may have to justify her decisions in front of somebody else—has been found by psychologists to strongly influence decision-making processes. The awareness of this issue remains however limited amongst economists, who tend to focus on the motivational effects of financial incentives. Accountability and incentives may provide different motivations for decision makers, and disentangling their effects is thus important for understanding real-world situations in which both are present. Separating accountability and incentives, I find different effects. Accountability is found to reduce preference reversals between frames, for which incentives have no effect. Incentives on the other hand are found to reduce risk seeking for losses, where accountability has no effect. In a choice task between simple and compound events, accountability increases the preference for the normatively superior simple event, while incentives have a weaker effect going in the opposite direction.