In the light of the discussions I noted in the previous post, I’ve been trying to get to grips with the thorny issue of metrics for researchers. A paper by Chris Armbruster of the Max Planck Society: Access, Usage and Citation Metrics: What Function for Digital Libraries and Repositories in Research Evaluation? has been really useful in providing an overview of the issues involved and the role that repositories can play in defining new ways of measuring effectiveness.
One of the things mentioned in this paper is a piece of software by Anne-Wil Harzing called Publish or Perish, which uses Google Scholar to search for materials by a particular author and then returns various measures of that individual’s impact in their field. The software is an alternative to using ISI’s citation index and is recommended for researchers in the social sciences, art and humanities and engineering disciplines, whose journals are more comprehensively covered by Google Scholar than ISI’s (according to the Publish or Perish website). The software returns a whole raft of ratings (many of which are explained in Armbruster’s article and all of which are covered in the software user’s manual). These are:
* Total number of papers
* Total number of citations
* Average number of citations per paper
* Average number of citations per author
* Average number of papers per author
* Hirsch’s h-index and related parameters, shown as h-index and Hirsch a=y.yy, m=z.zz in the output
* Egghe’s g-index, shown as g-index in the output
* The contemporary h-index, shown as hc-index and ac=y.yy in the output
* Two variations of the individual h-index, shown as hI-index and hI,norm in the output
* The age-weighted citation rate
* An analysis of the number of authors per paper.
I can see how this piece of software could get addictive if your chances for advancement are dependent upon these kinds of measures. And also how obsessive one might become about one’s rankings. My title for this post came from the mental picture I had of rival researchers jealously checking up on each other to see how their numbers compare. I’m not sure that this is healthy.
Researchers with unusual names are easy to track: searching for Herbert Van de Sompel’s research output is a doddle, for example, with no false hits being returned, as far as I could see (216 papers and 2,000 citations). However, searches for people who possess a combination of more common names, such as (for example) Paul Miller, bring back an impossible number of results, even when restricted to only one field of activity. A unique author identifier linked to his work would certainly be useful in Paul’s case, but I’m wondering if there isn’t some element of comfort in this current semi-anonymity for people with very common names.
Not that I’m trying to talk the Names Project out of a job, but I’d be interested to know if other people agree with Richard Hull’s point in a Times Higher Education article last week that having a number would make academics “prisoners of the research machine”. On balance I think I feel that most UK researchers are already caught up in the workings of the research machine, given the past and current focus on finding new ways of measuring their activity. Having unique researcher identifiers should make reporting and measuring the impact of published materials a lot easier, but the impact of this will be greater for some of us than it will be for our more uniquely-monikered colleagues.
Dorothea Salo, whose Caveat Lector blog is required reading for everyone involved in the institutional repository area, has written an article on the problems associated with the area of name authority in repositories of research outputs. This excellent overview of the issues faced by repository management is available (naturally) from the University of Wisconsin-Madison’s institutional repository.
Image by Francis M. McCoy, from Buttersweet on Flickr.
I’ll be presenting a paper on Names at the ISKO (International Society for Knowledge Organization) conference in Montréal in August. The paper had to be sent in to the organisers this week. It felt a bit strange writing a paper when development work on our prototype has only just started (our developer, Dan Needham, only joined Mimas last week). So I feel that the talk I give in August will probably bear little relation to the paper that was completed this week. It’s really an overview of where we are now and where we might be going in the next few months – there is a PDF version of the paper available now from the Names website.