“Looking-glass upon the wall, Who is fairest of us all?” (Part 1)

What is the impact of my research? This question has of course always been intriguing for scholars and scientists. For the bigger part of science history, it was a difficult question to answer. It was simply too difficult to know who was reading ones work, apart from the relatively small circle of direct colleagues. This has changed somewhat with the advent of the web and in particular with the exploding popularity of social media such as Facebook and Linkedin. Many of these tools do not only enable one to push ones work directly to “the world”, but also facilitate tracking some dimensions of its reception. So is it now possible to do immediate impact measurements of ones latest research publication?

This is the key question in a recent study we conducted in the framework of the SURFSHARE program of the Dutch universities, “Users, narcissism, and control”   (Wouters & Costas, 2012). As far as we know, we have done the first attempt at a comprehensive empirical analysis of the “Cambrian explosion of metrics” (Van Noorden, 2010) on the web. We collected detailed information about 16 recent publication impact monitors.

We draw the conclusion that these novel tools are quite interesting for individual researchers to have a quick (and sometimes dirty) impression of their possible impact. We call the tools used for this purpose “technologies of narcissism”. At the same time, it would be irresponsible to use these numbers in more formal research evaluations, to which we refer as “technologies of control” (Beniger, 1986). We therefore advise against already incorporating these tools in these more formalized settings because of the problems with data verification and control, and partly though less importantly, also because of the difficulties with normalizing the indicators to correct for field differences.

These new web based information services vary from enhanced peer review systems to tracking readers and downloads of publications to free citation services. For example, the Public Library of Science (PLoSOne) gives extensive user and citation statistics of articles published in PLoSOne. The new website Faculty of 1000 (F1000) brings together reviews and rankings of biomedical publications. This enables researchers to quickly zoom in on the stuff most relevant to their own research. Google Scholar, probably the most popular academic search engine, has recently started a new service Google Citations by which one can reconstruct ones citation impact on the basis of mentions in the Google Scholar database. Microsoft started a competing search engine with slightly different capabilities: Microsoft Academic Search. It gives more options through the use of APIs (application programming interfaces), so users can collect data with the help of scripts rather than having to do everything manually. An interesting development is also the creation of new types of reference managing software. For example, Mendeley is a combination of a social networking site and a reference manager. This enables scholars to efficiently exchange their reading tips and bibliographies. And this in its turn has created the possibility to know how many readers one has. These usage statistics are public. As a result, other services can harvest this data and represent them in a combined impact monitor. One of these new tools is TotalImpactwhich aims to present “the invisible impact” of scientific and scholarly articles on the basis of a document ID (such as a DOI or a URN). It harvests use data from a variety of sources, Mendeley being one of them.


Beniger, J. R. (1986). The Control Revolution. Technological and Economic Origins of the Information Society. Cambridge. Massachusets, and London, England: Harvard University Press.

Van Noorden, R. (2010). Metrics: A profusion of measures. Nature, 465(7300), 864-6. Nature Publishing Group. Retrieved from http://www.nature.com/news/2010/100616/full/465864a.html

Wouters, P., & Costas, R. (2012). Users , narcissism and control – tracking the impact of scholarly publications in the 21 century (pp. 1-50). Utrecht.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: