When social scientists disagree: comments on the Butler-van den Besselaar debate

In 2003, Linda Butler found evidence that as a result of “the increased culture of evaluation faced by the [Australian higher education] sector … [in which] significant funds are distributed to universities, and within universities, on the basis of aggregate publication counts, with little attention paid to the impact or quality of that output … journal publication productivity has increased significantly in the last decade [the 1990s], but its impact has declined” (Butler, 2003a, p.143). More recently, however, Peter van den Besselaar, Ulf Heyman and Ulf Sandström (hereafter BHS) have concluded from their bibliometric analysis that “Australia not only improved its share of research output but also increased research quality, implying that total impact was greatly increased”, and hence “Butler’s main conclusions are not correct” (van den Besselaar et al., 2017, p.1 ). How can we explain this disagreement?