Displacing impact factors with article-level measures of quality and expected impact in a peer-to-peer model of scholarly communication and evaluation

Author: Cooper Smout

Abstract: The academic community routinely uses journal impact factor (JIF) as a heuristic for article quality and expected impact, despite evidence that JIF is an unreliable measure of both traits. To break this status quo, it would be useful to develop article-level metrics that can predict future impact better than JIF. In this proof-of-concept study, experts will rate de-identified articles on various qualities of interest (e.g. novelty) and machine learning algorithms will be trained on these ratings to predict citation counts. I hypothesise that these impact algorithms will explain additional variance in the distribution of citation rates not accounted for by JIF. In future studies, these algorithms will be applied to newly published articles and fine-tuned as citation counts accrue.