In the light of the discussions I noted in the previous post, I’ve been trying to get to grips with the thorny issue of metrics for researchers. A paper by Chris Armbruster of the Max Planck Society: Access, Usage and Citation Metrics: What Function for Digital Libraries and Repositories in Research Evaluation? has been really useful in providing an overview of the issues involved and the role that repositories can play in defining new ways of measuring effectiveness.
One of the things mentioned in this paper is a piece of software by Anne-Wil Harzing called Publish or Perish, which uses Google Scholar to search for materials by a particular author and then returns various measures of that individual’s impact in their field. The software is an alternative to using ISI’s citation index and is recommended for researchers in the social sciences, art and humanities and engineering disciplines, whose journals are more comprehensively covered by Google Scholar than ISI’s (according to the Publish or Perish website). The software returns a whole raft of ratings (many of which are explained in Armbruster’s article and all of which are covered in the software user’s manual). These are:
* Total number of papers
* Total number of citations
* Average number of citations per paper
* Average number of citations per author
* Average number of papers per author
* Hirsch’s h-index and related parameters, shown as h-index and Hirsch a=y.yy, m=z.zz in the output
* Egghe’s g-index, shown as g-index in the output
* The contemporary h-index, shown as hc-index and ac=y.yy in the output
* Two variations of the individual h-index, shown as hI-index and hI,norm in the output
* The age-weighted citation rate
* An analysis of the number of authors per paper.
I can see how this piece of software could get addictive if your chances for advancement are dependent upon these kinds of measures. And also how obsessive one might become about one’s rankings. My title for this post came from the mental picture I had of rival researchers jealously checking up on each other to see how their numbers compare. I’m not sure that this is healthy.
Researchers with unusual names are easy to track: searching for Herbert Van de Sompel’s research output is a doddle, for example, with no false hits being returned, as far as I could see (216 papers and 2,000 citations). However, searches for people who possess a combination of more common names, such as (for example) Paul Miller, bring back an impossible number of results, even when restricted to only one field of activity. A unique author identifier linked to his work would certainly be useful in Paul’s case, but I’m wondering if there isn’t some element of comfort in this current semi-anonymity for people with very common names.
Not that I’m trying to talk the Names Project out of a job, but I’d be interested to know if other people agree with Richard Hull’s point in a Times Higher Education article last week that having a number would make academics “prisoners of the research machine”. On balance I think I feel that most UK researchers are already caught up in the workings of the research machine, given the past and current focus on finding new ways of measuring their activity. Having unique researcher identifiers should make reporting and measuring the impact of published materials a lot easier, but the impact of this will be greater for some of us than it will be for our more uniquely-monikered colleagues.
The issue of uniquely identifying researchers has been receiving a lot of attention in the past week. On Friday Chris Leonard described his vision for a unique author ID service at the PhysMath Central blog. On Tuesday Cameron Neylon posted about the possibility of having a specialist OpenID service for assigning unique identifiers to authors.* There is a lot of interesting follow-up discussion on this post at FriendFeed and a list of relevant resources on the OpenWetWare wiki. Both Andy Powell and Paul Walk have picked up on these items in recent posts. [Postscript: Thanks to Owen Stephens for pointing out that these discussions have been highlighted in a Times Higher Education article this week, too.]
Of course this discussion is all highly relevant and useful to us at the Names Project. We are currently developing a proposal for a continuation of the original Names Project, which would take the prototype we’ve been working on into a pilot authority system over the next two years. Funding permitting, this will allow us to investigate some of the suggestions that have been mooted in the discussions above.
*This was as a result of discussions following Björn Brembs’ presentation at the ScienceOnline09 conference on finding new ways of measuring the impact of researchers’ work.
Dorothea Salo, whose Caveat Lector blog is required reading for everyone involved in the institutional repository area, has written an article on the problems associated with the area of name authority in repositories of research outputs. This excellent overview of the issues faced by repository management is available (naturally) from the University of Wisconsin-Madison’s institutional repository.
Image by Francis M. McCoy, from Buttersweet on Flickr.