A recently released report by Council of Canadian Academies, Informing Research Choices: Indicators and Judgment, says that quantitative indicators, such as the number of publications and citation counts, cannot replace expert judgment when making decisions on allocating research grants.

Existing science assessment strategies can be categorized in many ways, including deliberative methods, such as peer review, and quantitative indicators, like publication and citation counts, numbers of researchers or students, research funding amounts, etc. “Quantitative indicators should be used to inform rather than replace expert judgment in the context of science assessment for research funding allocation.” The research panel reviewed the best practices worldwide and concluded that “the most promising strategies rely on a balanced use of quantitative indicators and expert judgment”.

The findings of this report will be used to devise a new budget allocation methodology for Natural Sciences and Engineering Research Council of Canada (NSERC)’s Discovery Grants Program. The current edition of Evaluation Indicators is available on NSERC’s website.

Adapted from Metrics can’t replace expert judgment in science assessments