To most people, the word ‘impact’ conjures up images of the forceful collision of one object into another, the landing of a blow, perhaps, or the crashing together of two cars moving at speed. In academia, impact has come to mean something rather different, although frequently no less concerning for those who struggle with it on a daily basis: the effect or influence of empirical work beyond academia. However, when we look at the ways in which impact is assessed and the systemic pressures placed on universities, journal editors, and researchers by this assessment, it is often something more like a car crash that comes to mind. Given the troubled relationship many academics have with the notion, it is worth stopping and asking ourselves just what is the actual impact of measuring impact? This editorial offers an explainer for the academic notion of impact and asks what it looks like for the Educational and Developmental Psychologist and other growing journals going forward.
What is impact?
As a researcher and academic interested in school belonging, the real impact of my work is found in the implementation by teachers of effective strategies and approaches that foster opportunities for their students to belong, in school leaders offering supportive climates for their staff, in communities working alongside schools to address a changing social landscape that impinges on feelings of belonging towards others and places, and, ultimately, in the sense of belonging felt by students towards their schools (Allen, Kern, Vella-Brodrick, Waters, & Hattie, Reference Allen, Kern, Vella-Brodrick, Waters and Hattie2018). Allen, Kern, Vella-Brodrick, & Waters, Reference Allen, Kern, Vella-Brodrick and Waters2017; Allen, Vella-Brodrick, & Waters, Reference Allen, Vella-Brodrick and Waters2016). However, to date, measures of impact do not directly assess the real-world implications of my research, but instead inform me if my publications are being cited by other researchers (not necessarily my central audience).
Traditional Journal Impact Factors (JIFS) are calculated by Clarivate Analytics using an extremely simple metric, based on nothing more than the number of times selected articles are cited. The more often a journal is cited, the higher its impact factor; the higher the impact factor, the more prestigious the journal is presumed to be. A number of other models incorporating a wider variety of metrics are also now used to evaluate journal rankings but all take the number of citations journals receive as their foundation. For example, SCImago Journal Rank (Scopus, Elsevier) evaluates scholarly journals based on number of citations, but weights these citations based on the ranking of the journals they are found in. The citations issued by more “prestigious” journals are thus of greater worth than citations from less important journals. The SNIP (Source Normalized Impact per Paper) and CiteScore ranking are other metrics that also draw on citation scores. Despite their widespread use, some scholarly work suggests that journal impact factors are ambiguous and unreliable (e.g., Casadevall & Fang, Reference Casadevall and Fang2015; Campanario, Carretero, Marangon, Molina, & Ros, Reference Campanario, Carretero, Marangon, Molina and Ros2011; Greenwood, Reference Greenwood2007).
For individual articles, Altmetric scores are becoming increasingly important. Altmetric scores measure how much attention an article receives online and are thus able to capture the degree of broader global interest in specific pieces of research rather than just the interest in the research shown by other researchers. These scores are inherently more inclusive, in that they widen the base of significant interested parties to take in all internet users rather than just other academics. As more people use technology as their primary vehicle for receiving information, Altmetric scores may, with time, become a reliable way to measure public engagement with research. Nevertheless, the actual impact of research remains elusive.
Why we need to talk about impact
Journal impact factors and other measures that are used to rank journals have an enormous influence on where researchers publish, on what they publish in order to reach their impact goals, and, ultimately, on who reads the research that they publish. Restrictions on where researchers can publish vary between universities and even countries, with some institutions being more inflexible than others. Publications in the more prestigious journals lead to rewards, incentives, and promotions. Never before has there been a more obvious systemic influence on researchers working in modern universities, a situation that can be compared to the exertion of overt political control over academia or even to past eras in which religious demands constrained research. This systemic hurdle has, itself, an enormous ‘impact’ (in the classical sense of the word), in that it arguably serves as a mechanism for controlling what information is generated by research and consequently disseminated to the general public.
As researchers, practitioners, and academics it is important that we discuss the notions of impact that have become prevalent in our institutions. Is impact measured effectively? And if it is (or perhaps is not), what are the consequences of measuring impact? While understanding the actual and real notable influence of research is important, it is wise to understand that high-quality and high-impact work can also be found outside of the more prestigious journals.
Towards real high-impact
The current issue of the Educational and Developmental Psychologists presents work with tremendous potential for applied and practical impact. With this in mind, how can we ensure that the articles in this current edition, Volume 37, Issue 2, have tangibly high impact? (See Alegre & Benson, Reference Alegre and Benson2019; Alonso & Little, Reference Alonso and Little2019; Declercq, Marle, & Pochon, Reference Declercq, Marle and Pochon2019; Jimeno, Reference Jimeno2019; Hemmings, Kay, & Sharp, Reference Hemmings, Kay and Sharp2019; Missenden & Campbell, Reference Missenden and Campbell2019; Wheldall, et al., Reference Wheldall, Bell, Wheldall, Madelaine, Reynolds and Arakelian2019). While citing is certainly to be encouraged, what is more important is that useful information is actually disseminated to those who can make valuable use of it. Circulate, share, and distribute; these are the best ways to ensure that research on educational and developmental psychology has real impact at the sharp end. Is there an interesting finding? An interesting quote? Then post about it on Twitter, Instagram, Linkedin or Facebook. Print the article out to hand around to interested parties in hard copy. Just make sure that the work researchers do gets into the hands of those who can use it.
Sharing information is the first step to take towards making a real impact. While Altmetric scores have sought to address the changing ways in which we measure impact, publication metrics are ultimately meaningless unless the research we produce leads to positive outcomes in the lives of others. To date, there is no genuine and reliable metric to measure this kind of real impact and perhaps there never will be. Or maybe the measure of impact should be as fluid and diverse as the actual research which an impact metric seeks to assess? Replication, evaluation, and validation then become alternative and yet, necessary ways in which impact can be measured. Although not an entirely new concept.
While the journal impact factor is undoubtedly here to stay and will continue to influence researcher behaviour, we should not confuse journal impact with journal importance. For professional journals such as the Educational and Developmental Psychologist, this creates an important distinction. Professional journals fill a scholarly gap that journals cannot meet by mere impact factor alone.
