Dutch reputation anxiety

 The recent Times Higher Education Top Universities by Reputation, published on 10 March 2011, has created some anxiety among Dutch universities. Some press releases suggested that this was a new ranking and it showed a much lower position of the universities than they had in the World Universities Ranking published in September 2010. To what extent should these universities worry?

 The recent reputation ranking is actually not a new ranking but the publication of a part of the older research underlying the September THE ranking. The reputation indicator that contributed to the ranking has now been published separately, which of course results in a different listing.

Comparing the two rankings, the reputation of the Dutch universities seems to be lower than their performance would justify. The Technical University Delft is highest at position 49. Among the top hundred only Utrecht University, Leiden University, and the University of Amsterdam are present. This contrasts clearly with the overall THE World Universities Ranking which is based not only on reputation but also on a mix of performance indicators. In that list, no less than ten Dutch universities are present among the best 200 universities of the world, with scores between 50 and 55 (Harvard scores 100). So this contrast might mean that the (relatively small) Dutch universities could improve their reputation management, especially at the international level.

 On the other hand, it is not clear how important this reputation ranking actually is. The results are based on an invitation only survey. THE sent out "tens of thousands" of requests to participate and received 13 thousand usable responses. It is unclear to what extent this sample is representative for the international academic community. There does appear to be some relation between the ranking results and effort in reputation management. The list is dominated by a small group of American universities together with Oxford University, so we see the usual suspects. All have invested in focused reputation management including the innovative use of new media. It would be interesting to analyze the determining factors for this reputation ranking. Perhaps THE can publish the underlying data?

Can metrics support open access monograph publishing?

Academic publishers are developing new ideas about open access in response to the economic crisis in scholarly publishing. To be sure, the open access movement was not started by publishers. It emerged from within the academic community in response to the serials crisis in university libraries and the rising prices of journals. Since 1995, open acccess has moved from the periphery to the centre of academia. The Berlin Declaration (2003) was an important moment that signaled that it was supported by most representatives of the scientific and scholarly establishment. Funding agencies are also moving towards open access as the prefered model as is the European Union, albeit slowly. This raises the question of how citation analysis and performance metrics can be developed that fit with this process. Metrics may either be used to monitor the growth and impact of open access publications. Metrics might also be used by researchers whose performance is evaluated to make the case that they contribute to the wider dissemination of their research.

A specific instance of open access publishing is the development of open access monographs. In this area, academic publishers are taking the lead, not coincidentally from smaller countries, prominent among them Amsterdam University Press. They are edged on by the rapid decline of the publication of printed scholarly books. This is partly caused by a shift from the production of monographs to articles in international peer reviewed journals in a number of fields in the social sciences and humanities. Also, book publishing is an expensive affair. If the sales numbers keep dropping it will soon become virtually impossible to publish scholarly monographs unless they also serve a much larger, non-specialist, audience. The problem is, of course, that book-length arguments are still important to synthesize research findings. This is not limited to the humanities or social sciences, but it is especially urgent in these fields. Can open access be (part of) the answer to this threat?

This was the topic of a conference organized by OAPEN on 25 February this year in Berlin. In my contribution to this conference, I have tried to sketch a possible research agenda to develop appropriate metrics for open access books. The issue of this type of metrics is a combination of four different measurement challenges. Each one of them is already a considerable problem: the measurement of open access (until now mostly developed for journals); the bibliometrics of the humanities and social sciences (plagued by the problem of inadequate source coverage by the Web of Science); webometrics of the use of information sources on the Web (confronted with enormous amounts of noise in the data); and the problem of measuring societal impact of research (often denoted with ‘societal quality’ – a problem of measuring widely diverging interactions).

If open access book metrics would simply be the sum of these four, it would probably be an impossible challenge to develop such a metrics. A research strategy would therefore need to focus on finding generic solutions that may be valid for more than one of these dimensions.

Currently, we simply do not have the tools to properly measure academic open acccess books. But we may use existing work as building blocks. My proposal is to follow a threefold strategy that works in parallel on three different tasks.

The first is to develop a model which describes how scholarly monographs are actually functioning in processes of the creation and use of research and scholarly knowledge. We already have some of these models focused on the economics of book publishing and these could be refined in terms of the information flows that the books are both an expression of and an element in.

The second task would be to work on data production and curation to ensure that the publication process captures the relevant data and that all parties agree on the crucial standards and technical norms needed. An prominent initiative is the COUNTER consortium, that aims to develop “an agreed international set of standards and protocols governing the recording and exchange of online usage data”. Our colleagues in Madrid have moreover developed a ranking method of Web based repositories that may be useful to build on in a metrics of open access books.

The third task is the creation of meaningful indicators of open access monographs. It may be useful to distinguish different dimensions: monitoring the creation or production of open access book sources; measuring their use by both scholars and lay audiences; and lastly their impact, implications, or influence on the creation and use of scientific and scholarly knowledge.

Anyone interested in joining forces in this endeavour?

%d bloggers like this: