Hostname: page-component-7b7567c8c-lpg2g Total loading time: 0 Render date: 2026-11-15T15:09:56.170Z Has data issue: false hasContentIssue false
  • English
  • Français

Evaluating University Research Performance Using Metrics

Published online by Cambridge University Press:  01 January 2026

Linda Butler and
Ian Mcallister
Linda Butler*
Affiliation:
Research School of Social Sciences, Australian National University, Canberra ACT 0200, Australia
Ian Mcallister*
Affiliation:
Research School of Social Sciences, Australian National University, Canberra ACT 0200, Australia
*
E-mail: linda.butler@anu.edu.au
E-mail: ian.mcallister@anu.edu.au
Get accessRights & Permissions [Opens in a new window]

Abstract

Evaluations of research quality in universities are now widely used in the advanced economies. The UK's Research Assessment Exercise (RAE), which began in 1986, is the most highly developed of these research evaluations. Based on peer review and involving some sixty-nine panels evaluating the research work of more than 50,000 academic staff, the exercise is expensive and time consuming. In this article, we examine the possibility that a quantitative, metrics-based approach can provide a low-cost alternative to expensive, qualitative peer review. To do this, we build on our previous work on political science by extending a metrics-based model to chemistry, using the results of the 2001 RAE. Our results show that no single model will apply across science and non-science disciplines. Any metrics approach to performance evaluation has to use a discipline-specific suite of indicators.

Keywords

research assessment exercisecitationschemistrypolitical scienceRAE

Information

Type
Profession
Information
European Political Science , Volume 10 , Issue 1 , March 2011, pp. 44 - 58
Copyright
Copyright © 2011 European Consortium for Political Research

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Article purchase

Temporarily unavailable

Footnotes

*

Corresponding author.

References

Arts and Humanities Research Council and the Higher Education Funding Council of England (AHRC and HEFCE). (2006) ‘Use of research metrics in the arts and humanities’, available at http://www.hefce.ac.uk/research/ref/group/, accessed 3 December 2008.Google Scholar
Bence, V. and Oppenheim, C., (2005) The evolution of the UK's research assessment exercise: publications, performance and perceptions, Journal of Educational Administration and History, 37, 137–155.CrossRefGoogle Scholar
Butler, L. and McAllister, I., (2009) Metrics or peer review? Evaluating the 2001 UK research assessment exercise in political science, Political Studies Review, 7, 3–17.CrossRefGoogle Scholar
Buzan, B., Wæver, O. and de Wilde, J. (1998), Security: A New Framework for Analysis, Boulder, Co/London: Lynne ReinerGoogle Scholar
Council for the Humanities, Arts and Social Sciences (CHASS). (2005) ‘Measures of quality and impact of publicly-funded research in the humanities, arts and social sciences’, available at http://www.chass.org.au/papers/PAP20051101JP.php, accessed 3 December 2008.Google Scholar
Dixon, H. and Suckling, J., (1996) Measuring Outcomes in Higher Education, Measuring Outcome in the Public Sector, London: Taylor and Francis, pp. 47–56Google Scholar
Department for Education and Skills (DfES). (2006) ‘DfES consultation on the reform of higher education research assessment and funding: summary of responses’, available at: http://www.dfes.gov.uk/consultations/conResults.cfm?consultationId=1404, accessed 3 December 2008.Google Scholar
Heinrich, C.J., (2002) Outcomes-based performance management in the public sector: implications for government accountability and effectiveness, Public Administration Review, 62: 712–725.CrossRefGoogle Scholar
Hirsch, J.E., (2006) An index to quantify an individual's scientific research output, Proceedings of the National Academy of Science, 102(46): 16569–16572.CrossRefGoogle Scholar
HM Treasury, Department of Trade and Industry, Department for Education and Skills, and Department of Health. (2006) ‘Science and innovation investment framework 2004–2014: next steps’, available at http://www.berr.gov.uk/dius/science/science-funding/framework/next_steps/page28988.html, accessed 3 December 2008.Google Scholar
Jackman, R.W. and Siverson, R.M., (1996) Rating the rating: an analysis of the national research council's appraisal of political science Ph.D. programs, PS: Political Science and Politics, 29: 155–160.Google Scholar
Moed, H.F. (2005), Citation Analysis in Research Evaluation, Heidelberg: SpringerGoogle Scholar
Mouffe, C. (2000), The Democratic Paradox, London: VersoGoogle Scholar
Propper, C. and Wilson, D., (2003) The use and usefulness of performance measures in the public sector, Oxford Review of Economic Policy, 19: 250–267.CrossRefGoogle Scholar
Research Assessment Exercise (RAE). (2001a) ‘Panels’ criteria and working methods. 3.12 chemistry UoA 18’, available at http://www.rae.ac.uk/2001/Pubs/5_99/ByUoA/crit18.htm, accessed 3 December 2008.Google Scholar
Research Assessment Exercise (RAE). (2001b) ‘Panels’ criteria and working methods. 3.31 politics and international studies, UoA 39’, available at http://www.rae.ac.uk/2001/Pubs/5_99/ByUoA/crit39.htm, accessed 3 December 2008.Google Scholar
Research Assessment Exercise (RAE). (2008) ‘Research assessment exercise 2008: The outcome’, available at http://www.rae.ac.uk/pubs/2008/01/, accessed 25 December 2008.Google Scholar
UK Parliament. (2002) ‘Memorandum submitted by the higher education funding council for England (HEFCE)’, available at http://www.parliament.the-stationery-office.co.uk/pa/cm200102/cmselect/cmsctech/507/2012302.htm#n7, accessed 16 January, 2009.Google Scholar