Allow content?
This content requires cookies. To view content please update your cookie preferences.
Allow content?
This content requires cookies. To view content please update your cookie preferences.
EJPR Guidelines for Research Reproducibility
The following guidelines set out the requirements for the replication of all empirical analyses reported in manuscripts accepted to the European Journal of Political Research (EJPR). They distinguish between required and recommended procedures. Only the required procedures are mandatory; compliance with recommended procedures will not affect the likelihood of publication.
Please read these guidelines carefully. Upon submission, you will be asked to confirm that you agree to comply with the required procedures if your manuscript is conditionally accepted. You may request an exemption from specific requirements at the time of submission.
Allow content?
This content requires cookies. To view content please update your cookie preferences.
1. Guiding Principles & Objectives
EJPR is committed to the highest standards of research transparency and integrity. Following this logic, our reproducibility policy is designed to achieve three core objectives:
- Transparency: The procedure(s) used to collect and analyse data, as well as the data itself, should be available where possible to enable future researchers to build upon, critique, and extend the published work. This applies to quantitative and qualitative research.
- Comprehensibility: Readers with the appropriate technical knowledge can understand how a paper's findings were produced by reviewing the available documentation.
- Quality Control: Empirical results should be computationally sound and error-free.
To comply with this policy, authors of accepted articles must submit a reproducibility package as a condition of acceptance.
Allow content?
This content requires cookies. To view content please update your cookie preferences.
2. Scope of Reproducibility
The reproducibility package contains all materials necessary to reproduce every computational result reported in the manuscript and its supplementary materials. This includes all tables, figures, and numerical results mentioned in the text.
The package must include:
- A README file
- The clean data files and their documentation
- All code files required to run the final analyses and generate the tables and figures
- Output files.
We also recommend sharing:
- The original, unprocessed data files
- All code required to clean, process, merge, and transform the raw data into the final analysis dataset
- The original data collection’s additional documentation.
For studies involving original data collection (e.g., surveys, experiments, web scraping), the replication package should describe how the data was collected. This includes the criteria for participant eligibility and selection; all data collection instruments, such as survey questionnaires, experimental protocols, or scraping scripts; and the software or scripts used (e.g., z-Tree code, survey structure of a LimeSurvey, Qualtrics files where possible).
For simulations, the package must include all code needed to reproduce the results from scratch. To enable exact replication, simulation-based analyses should also specify all random seed values.
We recognise that sharing raw data is not always possible for practical or legal reasons. Our guidelines therefore aim to balance transparency with flexibility by encouraging practical solutions that support, rather than constrain, researchers.
Allow content?
This content requires cookies. To view content please update your cookie preferences.
3. The Submission Process: Data Availability, Transparency, and Archiving
These guidelines are designed to minimise the burden on researchers submitting to EJPR. Compliance is required only if a manuscript is conditionally accepted.
- Policy Agreement: Upon initial submission, authors confirm their willingness to comply with this policy or, in rare cases, request an exemption. This happens within the submission platform. EJPR will not accept a separate statement within the manuscript.
- Package Submission: Authors must submit a complete reproducibility package to the EJPR Dataverse after conditional acceptance. The journal will review the completeness of the package before granting final acceptance.
- Archiving/Repository: Reproducibility packages are deposited in the dedicated EJPR Dataverse hosted on Harvard Dataverse, which provides a persistent digital object identifier (DOI), is citable, and includes versioning.
- Data Availability Statement (DAS): The final manuscript includes a ‘Data Availability Statement (DAS)’ with the full citation and DOI for the reproducibility package. In cases where the journal has granted an exemption to this policy, the DAS should explain why any code or data could not be included in the package. EJPR does not consider ‘Data available upon request’ an acceptable access condition if contingent on authors’ approval.
- Pre-registrations: If applicable, the manuscript should correctly reference the pre-registration, using a DOI or other persistent identifier, and clearly link to it in the README file.
Allow content?
This content requires cookies. To view content please update your cookie preferences.
4. Exemptions
These requirements apply to all articles that rely on statistical or computational methods. This includes quantitative analysis as well as qualitative research that uses computational techniques (e.g. automated content analysis, Qualitative Comparative Analysis) where data are organised in a matrix form.
Articles that do not involve statistical or computational analysis, including entirely theoretical or conceptual contributions, may be exempt from the reproducibility requirements. Authors may also apply for an exemption where data or other research materials cannot reasonably be shared, for example because of ethical, legal, confidentiality, or data protection considerations.
We nevertheless strongly encourage all authors to share research materials where appropriate and to make conclusions based on qualitative interpretation as accessible and transparent as possible.
For theoretical and conceptual work, this might include relevant research packages, coding or analytical frameworks, supplementary materials, or other documentation that helps readers understand and assess how the article's conclusions were reached.
Allow content?
This content requires cookies. To view content please update your cookie preferences.
5. Components of the Reproducibility Package
A reproducibility package should be self-contained, allowing a third party to reproduce all results without contacting the authors. All files should be downloadable and usable without complex setup and organised in a logical structure, with separate folders for data, code, and results, accompanied by clear documentation. The package must include:
Allow content?
This content requires cookies. To view content please update your cookie preferences.
A. The README File
This is the central ‘map’ to your package and should be the first file a user opens. The core aspects of the reproducibility package, such as the names and roles of all files, the order of scripts and other relevant information, must be clear. The README file must be in a non-proprietary format (e.g., .txt or .md), in the root directory.
It must contain:
- Table of Contents: A brief description of every file and folder in the package, explaining its purpose.
- Data Provenance: A clear description of all data files, including how and where each was originally acquired. For secondary data, provide citations and links.
- Step-by-Step Instructions: A clear guide on how to run the code to reproduce all results, and a reference to the master script (see below).
- Output Map: A list stating explicitly which script/code section generates each table and figure in the manuscript and the appendix.
- Information on the Computational Environment:
a. Operating System (e.g., Windows 11, macOS Sonoma 14.0, Ubuntu 20.04).
b. Software used and its exact version number (e.g., R 4.4.1, Stata 18).
c. A complete list of all required packages, libraries, or add-ons with their version numbers.
- Runtime Estimate: Note any steps that are particularly time-consuming.
a. We also recommend including an estimate of how long the entire code took to run.
- Random Seeds: For any analysis involving stochastic processes (e.g., simulation, bootstrapping), you must set a random seed in your code and state in the README where the seed is set.
Allow content?
This content requires cookies. To view content please update your cookie preferences.
B. Data Files
- Analysis Data: Each dataset must include the final ‘analysis-ready’ dataset.
- We also recommend sharing the raw data, if legally and practically possible (see optional additional files below).
- Data Documentation (Codebook): Every dataset must be accompanied by a codebook that describes each variable used, its values, and its units of measurement. If using a public dataset with existing documentation, refer to this documentation in the README.
- Data Citation: All datasets used must be formally cited in the manuscript's reference list.
- Format: Provide data in an open, non-proprietary format (e.g., .csv, .tsv) whenever possible.
Allow content?
This content requires cookies. To view content please update your cookie preferences.
C. Code Files
- Master Script: If the analysis involves more than one script, provide a single master script that calls and executes all other scripts in the correct order (e.g., master.R, run_all.sh). The entire analysis should be executable with a single command. If using multiple statistical software (such as R, Stata, …), provide one master script per statistical software.
- Individual scripts: Ideally, label individual scripts in order (e.g., 0_master.R, 1_data_cleaning.R, 2_analysis.R) and store in one subfolder (such as ./scripts/).
- Clarity and Documentation:
- Embrace Literate Programming: We encourage using literate programming tools like Quarto, R Markdown or Jupyter Notebooks. These tools interweave your executable code with narrative text, figures, and tables, creating a single document that explains your research process step by step. These tools can also create runtime estimates and information on the computational environment.
- Code must be well-commented to help others understand the steps of the analysis. Use descriptive names for files and variables.
- No Absolute Paths: Code must use relative paths (e.g., ../data/dataset.csv), not absolute paths (e.g., C:/Users/MyName/…).
- No Manual Input: Scripts should run from start to finish without manual user input.
Allow content?
This content requires cookies. To view content please update your cookie preferences.
D. Output Files
Your code must save every table and figure as a separate, clearly named file in a results or output folder (e.g., figure1.pdf, table2.tex). These files must be included in the reproducibility package to reflect the full state of the computational environment and all its outputs.
Optional additional files:
- Raw and Analysis Data: Include the ‘raw’ data as originally obtained and, if the cleaning process is computationally intensive, the final ‘analysis-ready’ dataset.
Allow content?
This content requires cookies. To view content please update your cookie preferences.
6. Recommendation: Submission via a Computational Container
To minimise problems related to software versions and dependencies, we recommend submitting a containerised reproducibility package using tools such as Docker or Binder. Containers allow researchers to reproduce the computational environment and results across different computers at any time, ensuring long-term transparency and usability.
Authors who submit a package with a containerised environment may be included in a Docker Special Collection.
Allow content?
This content requires cookies. To view content please update your cookie preferences.
7. Restricted Data and Non-Empirical Analyses
Sharing raw data is not always possible for practical, ethical, or legal reasons. Authors working with restricted data should indicate this at the time of initial submission. In such cases, authors should provide a reproducibility package that maximises transparency, including:
- All analysis code used to generate the reported results.
- A detailed ‘pseudo-codebook’ describing the restricted data and relevant variables.
- Instructions on how another researcher could apply for and obtain access to the data.
- A log file showing the complete output generated by running the analysis code on the restricted data.
EJPR encourages authors to use data anonymisation and other privacy-preserving techniques. If data cannot be deposited in a public repository, authors should preserve their data and code for at least five years following publication and provide reasonable assistance with replication requests. ‘Data available upon request’ is insufficient where access depends solely on approval by the authors.
For theoretical articles that do not involve original data analysis, a reproducibility package may not be necessary. In such cases, authors should provide any materials that support transparency, such as formal proofs, computational models, simulations, or code used to generate theoretical results.
EJPR encourages the community to run reproducibility packages. The editorial office will check only that the package is compliant. We will not run reproducibility packages to output files; it remains the author’s responsibility to ensure the accuracy of submitted files.