Hostname: page-component-74656f9884-rngvg Total loading time: 0 Render date: 2026-11-05T09:28:37.668Z Has data issue: false hasContentIssue false
  • English
  • Français

Big data meets open political science: an empirical assessment of transparency standards 2008–2019

Published online by Cambridge University Press:  01 January 2026

Karin Dyrstad  and
Jonathon W. Moses
Karin Dyrstad*
Affiliation:
Department of Sociology and Political Science, Norwegian University of Science and Technology, 7491 Dragvoll, Trondheim, Norway
Jonathon W. Moses*
Affiliation:
Department of Sociology and Political Science, Norwegian University of Science and Technology, 7491 Dragvoll, Trondheim, Norway
*
E-mail: karin.dyrstad@ntnu.no
E-mail: karin.dyrstad@ntnu.no
Get access Rights & Permissions [Opens in a new window]

Abstract

Over the last decade, the field of political science has been exposed to two concomitant developments: a surge of Big Data (BD) and a growing demand for transparency. To date, however, we do not know the extent to which these two developments are compatible with one another. The purpose of this article is to assess, empirically, the extent to which BD political science (broadly defined) adheres to established norms of transparency in the discipline. To address this question, we develop an original dataset of 1555 articles drawn from the Web of Science database covering the period 2008–2019. In doing so, we also provide an assessment of the current level of transparency in empirical political science and quantitative political science in general. We find that articles using Big Data are significantly less likely than other, more traditional works of political science, to share replication files. Our study also illustrates some of the promises and challenges associated with extracting data from Web of Science and similar databases.

Keywords

Big dataTransparencyReplicationDARTSocial media

Information

Type
Research
Information
European Political Science , Volume 22 , Issue 2 , June 2023 , pp. 182 - 201
Copyright
Copyright © 2022 European Consortium for Political Research

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Article purchase

Temporarily unavailable

Footnotes

Copyright comment

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

References

Anderson, C. 2008. The end of theory: The data deluge makes the scientific method obsolete. Wired, 23 June.Google Scholar
Berghel, H. (2018) Malice domestic: The Cambridge analytica dystopia Computer 51 (5): 84–89.CrossRefGoogle Scholar
Bethlehem, J. (2017) The representativity of election polls Statistics, Politics and Policy 8: 1–12.CrossRefGoogle Scholar
Blumenstock, J., Cadamuro, G. and On, R. (2015) Predicting poverty and wealth from mobile phone metadata Science 350 (6264): 1073–1076.CrossRefGoogle ScholarPubMed
Broockman, D., J. Kalla and P. Aronow. 2015. Irregularities in LaCour (2014). 19 May. Available from: http://stanford.edu/~dbroock/broockman_kalla_aronow_lg_irregularities.pdf. (accessed 14 March 2018).Google Scholar
Bueno de Mesquita, B., Gleditsch, N.P., James, P., King, G., Metelits, C., Ray, J.L., Russett, B., Strand, H. and Valeriano, B. (2003) Symposium on replication in international studies research International Studies Perspectives 4: 72–107.CrossRefGoogle Scholar
Bull, M.J. (2016) Introduction; open access in the social and political sciences: Threat or opportunity? EPS 15: 151–157.Google Scholar
Bump, P. 2020. It’s important to ask why 2020 polls were off. It’s more important to ask what will happen next. The Washington Post, 16 November.Google Scholar
Burgess, J. and A. Bruns. 2012. Twitter archives and the challenges of ‘Big Social Data’ for media and communication research. M/C Journal 15(5).CrossRefGoogle Scholar
Cohen, N. 2020. What went wrong with Polling? Some early theories. The New York Times, 10 November.Google Scholar
Cooper, C., Collins, T.A. and Knotts, H.G. (2009) Picturing political science PS 42: 365–365.Google Scholar
Coppock, A. (2017) Did Shy Trump Supporters Bias the 2016 Polls? Evidence from a Nationally-representative List Experiment Statistics, Politics and Policy 8: 29–40 10.1515/spp-2016-0005.CrossRefGoogle Scholar
Dalton, C. and J. Thatcher. 2015. Inflated granularity: Spatial ‘Big Data’ and geodemographics. Big Data & Society (July-December): 1–15.CrossRefGoogle Scholar
D'Ignazio, C. and Klein, L.F. (2020) Data Feminism Cambridge: MIT Press.CrossRefGoogle Scholar
Doctorow, C. (2008) Big data: Welcome to the petacenter Nature 455: 16–21.CrossRefGoogle Scholar
Dommett, K. and Tromble, R. (2022) Advocating for platform data access: Challenges and opportunities for academics seeking policy change Politics and Governance 10 (1): 220–229.CrossRefGoogle Scholar
Elman, C. and D. Kapiszewski. 2014. Data access and research transparency in the qualitative tradition. PS 47: 43–47.CrossRefGoogle Scholar
Elman, C., Kapiszewski, D. and Lupia, A. (2018) Transparent social inquiry: Implications for political science Annual Review of Political Science 21: 29–47.CrossRefGoogle Scholar
Giles, M.W. and J.C. Garrand. 2007. Ranking Political Science Journals: Reputational and Citational Approaches. PS 40: 741–51.CrossRefGoogle Scholar
Gleditsch, N.P. and Metelits, C. (2003) Symposium on replication in international studies research International Studies Perspectives 4: 72–107.Google Scholar
Herndon, T., M. Ash and R. Pollin. 2013. Does High Public Debt Consistently Stifle Economic Growth? A Critique of Reinhart and Rogoff, April 15. PERI Working Paper No. 322.Google Scholar
Iacus, S.M., King, G. and Porro, G. (2012) Causal inference without balance checking: Coarsened exact matching Political Analysis 20: 1–24 10.1093/pan/mpr013.CrossRefGoogle Scholar
Janz, N. 2018. Replication and transparency in political science—did we make any progress? Political Science Replication blog. 14 July. Available from: https://politicalsciencereplication.wordpress.com/2018/07/14/replication-and-transparency-in-political-science-did-we-make-any-progress/ (accessed 15 June 2022).Google Scholar
Jensen, M.R. and Moses, J.W. (2021) The state of political science, 2020 EPS 20: 14–33.Google Scholar
King, G. 1995. Replication, Replication. PS 28: 444–452.Google Scholar
LaCour, M.J. and Green, D.P. (2014) When contact changes minds: An experiment on transmission of support for gay equality Science 346: 1366–1369.CrossRefGoogle ScholarPubMed
Laitin, D.D. and R. Reich. 2017. Trust, transparency, and replication in political science. PS 50: 172–175.Google Scholar
Laney, D. 2001. 3D data management: Controlling data volume, velocity, variety. Application Delivery Strategies Meta Group File 949. Available from: http://blogs.gartner.com/doug-laney/files/2012/01/ad949-3D-Data-Management-Controlling-Data-Volume-Velocity-and-Variety.pdf (accessed 14 May 2015).Google Scholar
Lax, J.R. and Phillips, J.H. and Stollwerk, A.F. (2016) Are survey respondents lying about their support for same-sex marriage? Lessons from a list experiment Public Opinion Quarterly 80: 510–533.CrossRefGoogle ScholarPubMed
Longley, P. (2012) Geodemographics and the practices of geographic information science International Journal of Geographical Information Science 26: 2227–2237.CrossRefGoogle Scholar
Lupia, A. and C. Elman. 2014. Openness in political science: data access and research transparency—introduction. PS 47: 19–42.CrossRefGoogle Scholar
May, C. (2005) The academy's new electronic order? Open source journals and publishing political science EPS 4: 14–24.Google Scholar
Miguel, E., Camerer, C., Casey, K., Cohen, J., Esterling, K.M., Gerber, A. and Laitin, D. (2014) Promoting transparency in social science research Science 343: 30–31.CrossRefGoogle ScholarPubMed
Millward, D. 2016. How the pollsters got the US election wrong—just like Brexit. The Telegraph, 9 November.Google Scholar
Moher, D., Liberati, A., Tetzlaff, J. and Altman, D.G. (2009) Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement PLOS Medicine 6: e1000097.CrossRefGoogle ScholarPubMed
Powell, B. et al. 2016. Letter from distinguished political scientists urging nuanced journal interpretation of JETS policy guidelines. 13 January. Available from: https://politicalsciencenow.com/letter-from-distinguished-political-scientists-urging-nuanced-journal-interpretation-of-jets-policy-guidelines/ (accessed 9 December 2020).Google Scholar
Reinhart, C. and Rogoff, K.K. (2010) Growth in a Time of Debt American Economic Review: Papers & Proceedings 100: 573–578 10.1257/aer.100.2.573.CrossRefGoogle Scholar
Singal, J. 2015. The Case of the Amazing Gay-Marriage Data: How a Graduate Student Reluctantly Uncovered a Huge Scientific Fraud. Science of Us blog. 29 May. Available from: http://nymag.com/scienceofus/2015/05/how-a-grad-student-uncovered-a-huge-fraud.html (accessed on 14 March 2018).Google Scholar
Tromble, R. 2021. Where Have All the Data Gone? A Critical Reflection on Academic Digital Research in the Post-API Age. Social Media and Society 1–8.CrossRefGoogle Scholar
Tufekci, Z. 2020. Can we finally agree to ignore election forecasts? The New York Times, 1 November.Google Scholar
Wylie, C. (2019) Mindf*ck: Cambridge Analytica and the plot to break America New York: Random House.Google Scholar
Supplementary material: File

Dyrstad and Moses supplementary material

Online Supplemental Material
Download Dyrstad and Moses supplementary material(File)
File 134 KB