JIFs, giraffes, and a diffusion of culpability: A response to Osterloh and Frey's discussion paper on ‘Borrowed plumes’

The giraffe's laryngeal nerve is an absurdity. It connects the brain to the voice box, taking a circuitous route all the way down the neck and all the way back up again. Presumably, somewhere in the course of history, it got caught up with the development of other organs, and now takes a massive detour round the back of the aorta near the heart. It is anything but elegant, and was clearly “not meant to be, but just happens to be” (Gould, 1993, p76). Disentangling the nerve would involve the rearrangement of multiple organs simultaneously, and would be – to use policy parlance – a multi-stakeholder issue. The journal impact factor (JIF) is an absurdity. It connects two weak bodies of understanding – citation theory and peer review theory – via a circuitous route through the research system. The JIF was supposed to assist librarians in curating their collection, and the underlying Citation Index1 itself was supposed to help with searches and retrieval, but somewhere in the course of history, they both got caught up in an information flood. They are now used to rank selectively, a purpose for which they were certainly not designed. JIFs have ended up deeply entangled in modern research systems, and implicated in important pathologies of those systems. Of course, the nerve remains functional for most giraffes, which is more than can be said about JIFs in the research system, so the metaphor is a strained one. Even a cursory look at the JIF reveals many reasons why it does not serve us well for ranking journals. It is a metric that is unforgiving of disciplinary differences and journal styles, and easily gamed by strategic authors, editors and publishers, to name just some of the issues (Archambault and Larivière, 2009; Braun, 2012; Martin, 2016). Perhaps most troubling is the idea that journal rankings may now be shaping the direction and content of science (Rafols et al., 2012; Muller and Rijke, 2017). Authors might complain about what is needed to get published in high-JIF journals, whilst editors might lament that submissions are becoming more homogenous. Journals certainly seem to shape and influence researchers’ behaviour. Economists might not be willing to give their right arm for a publication in American Economic Review with its lofty stature, but the strength of their preferences imply they would at least be willing to sacrifice more than half a thumb (Attema et al., 2014)!2 Yet, the metaphor does serve to illustrate a choice for addressing the current malaise in research policy. Should we ‘muddle through’, à la Lindblom (1959), tinkering with modified JIFs, using them in combination with other metrics, playing the arms race with those who seek to game them?3 Or should we seek radical disentanglement from JIF, or indeed from journal rankings by any measure? Within this context, Osterloh and Frey (2020) do us an important service by asking why, for all its problems, journal rankings based on JIFs are still so influential. Moreover, they offer some suggestions for reform, some more radical than others. In this note, I wish to focus on two points of difference between Osterloh and Frey, and myself. The first is the degree to which JIF lock-in can be explained exclusively by ‘borrowed plumes’. The second is the degree to which their suggested reforms are sufficiently radical but too narrow in scope. Considering these two points, which turn out to be somewhat intertwined, will help to characterise what is really at stake. What is at stake is not so much how journal rankings affect academics and their careers (“top publications are decisive for academic careers”), but rather how well the research system interacts with and serves societal goals. Osterloh and Frey may well have identified one source of JIF lock-in, but I shall submit that there are other culprits, too, both within and beyond academia. Some are more culpable than others, readers may decide.