First, I put no more faith in this predictor than I would in astrology. The h-index predictor ought to come with the sort of disclaimer that skeptics asked newspapers to add to horoscopes: “purely for entertainment purposes.”
Justin Kiggins made the best critique of the h-index predictor:
according to the H-Index predictor, my cat will have an H-Index of 9 in 2022
So in ten years, Justin’s cat will have a higher h-index than I do after 20 years of publishing science. In fairness, though, this is Justin’s cat:
More seriously, I was a little surprised by some of the discussions I saw around the Nature paper (Acuna et al. 2012). For instance, this news article botches the basic idea of the article.
But the new formula is more than twice as accurate as the h index for predicting future success for researchers in the life sciences.
It’s not “more accurate” than the h-index, it is the h-index. It’s just a way to predict h-index out into the future, when h-index is a calculation based on what you’ve done in the past.
There’s a few other things that are worth pointing out about h-index.
First, the whole point of h-index is that it is supposed to give some indication of scientific quality or reputation, and not reward people for publishing rubbish papers. Someone who publishes a lot of papers that nobody cares about or cites should have a low h-index. The theory is sound, but a recent paper found that the number of papers you’ve written explains 87% of the variation in h-index (Gaster and Gaster 2012).
In a way, this is reassuring. This suggests that most scientists are competent, and most papers are worth citing. But it suggests that paper quality is not very important to this measure.
Another issue is reproducibility. For instance, my h-index according to Google Scholar is 8. According to Web of Science, it’s 6. There is not way that h-indices are going to be calculated by hand by human beings; they must be done by computers running algorithms. And if different datasets give the different results, it’s limited in its usefulness.
Measures like h-index this are entertaining, and as I’ve said before, they are expedient. But the moment you hear any administrator seriously suggesting using these alone, in isolation, to make important decisions like tenure and promotion, run. Or fight. But don’t just accept it.
Gaster N, Gaster M. 2012. A critical assessment of the h-index. BioEssays 34(10): 832. DOI: 10.1002/bies.201200036
Acuna DE, Allesina S, Kording KP. 2012. Future impact: Predicting scientific success, Nature 489(7415): 202. DOI: 10.1038/489201a
Gazing into the crystal ball of h-index
The Genius Index: One Scientist's Crusade to Rewrite Reputation Rules