New CWTS blog

Dear readers,

It is with great pleasure that we announce a new platform for blog posts emanating from our institute: the CWTS blog.

How important is the number of citations to my scientific work really? How is evaluation influencing knowledge production? Should my organisation support the DORA declaration? When does it (not) make sense to use the h-index? What does a competitive yet conscientious career system look like? What is the relation between scientific and social impact of research? How can we value diversity in scholarship?

The CWTS blog brings together ideas, commentary, and (book) reviews about the latest developments in scientometrics, research evaluation, and research management. It is written for those interested in bibliometric and scientometric indicators and tools, implications of monitoring, measuring, and managing research, and the potential of quantitative and qualitative methods for understanding the dynamics of scientific research.

This a moderated blog with a small editorial team consisting of Sarah de Rijcke, Ludo Waltman, and Paul Wouters. The blog posts are written by researchers affiliated to CWTS.

In the meantime, this current Citation Culture blog will be discontinued. Thank you all very much for your dedicated readership. We hope you will enjoy reading the new blog!

You can subscribe to the mailinglist or rss feed at www.cwts.nl/blog

Advertisement

NWO president Jos Engelen calls for in-depth study of editorial policies of Science and Nature

The Netherlands Organization for Scientific Research (NWO) wants to start an in-depth study of the editorial policies of the most famous scientific journals, such as Science, Nature, Cell, The Lancet, The New England Journal of Medicine, and Brain. NWO president Jos Engelen announced this in a lecture on open access publishing on 11 December in Nijmegen. The lecture was given in the framework of the Honors Program of Radboud University on “Ethos in Science”.

According to Engelen, it is urgent to assess the role of these journals in the communication of scientific knowledge. Engelen wants the scientific system to shift to free dissemination of all scientific results. He sees three reasons for this. First, it is “a moral obligation to grant members of the public free access to scientific results that were obtained through public funding, through taxpayers’ money.” Engelen gets “particularly irritated when I read in my newspaper that new scientific results have been published on, say, sea level rise, to find out that I have to buy the latest issue of Nature Magazine to be properly informed.” Second, scientific knowledge gives a competitive edge to the knowledge economy and should therefore freely flow into society and the private sector. Third, science itself will profit from the free flow of knowledge between fields. “In order to face the ‘grand challenges’ of today scientific disciplines have to cooperate and new disciplines will emerge.”

Engelen wants to investigate the editorial policies of the most famous scientific journals because they stand in the way of open access. These feel no reason to shift their business model to open access, because their position is practically impregnable”. Engelen takes the journal Science, published by the Association for the Advancement of Science as example. “Its reputation is based on an extremely selective publishing policy and its reputation has turned ‘Science’ into a brand that sells”. Engelen remarks that the same is true for Nature, Cell and other journals published by commercial publishers. “Scientific publications are only a part, not even the dominant part of ‘the business’, but the reputation of the journal is entirely based on innovative science emanating from publicly funded research. Conversely, the reputation of scientists is greatly boosted by publications in these top-journals; top-journals with primarily an interest in selling and not in, for example, promoting the best researchers.”

Engelen concludes this part of his lecture on open access with a clear shot across the bow. “It has puzzled me for a while already that national funding organisations are not more critical about the authority that is almost automatically imputed to the (in some cases full time, professional, paid) editors of the top-journals. I think an in depth, objective study of the editorial policies, and the results thereof, commissioned by research funders, is highly desirable and in fact overdue. I intend to take initiatives along this line soon!”

Anxiety about quality may hinder open access

Anxiety about the quality of open access journals hinders the further spread of open access publications. This conclusion was cited many times during the recent Co-ordinating workshop on Open Access to Scientific Information, in Brussels on May 4 this year. The workshop was attended by about 70 key players in Open Access and was organized by two EU directorates: Research and Information Society & Media. The critical role of quality control came to the fore in various ways.

Salvatore Mele (CERN), coordinator of the SOAP project presented the results of their study (based on a Web survey) of the attitudes prevailing among researchers with respect to open access. They reveal a remarkable gap between strong support for open access on the one hand and a lack of actual open access publishing on the other hand. 89 % of the researchers say they are in favour of open access publishing. At the same time, only between 8 and 10 % of the articles published are open access. According to the SOAP study, two factors are mainly responsible for this gap: the problem of financing open access publications and the perceived lack of quality of many open access journals. The Journal Impact Factor of journals was also mentioned as a reason not to publish in existing open access journals.

The weight of these factors does vary by field. For example, in chemistry 60 % of the researchers mention financial reasons as barrier to open access, whereas only 16 % of the astronomers see finance as problematic. In astronomy, worries about the quality of journals are mentioned most (by more than half of the astronomers) whereas this is only seen as a problem by about one-fifth of the chemists. This result points, by the way, to the need to develop specific open access policies for different scientific and scholarly fields. For example, in the humanities open access books will be an important issue.

Quality of the journals was also central in a new initiative made public at the workshop by the delegation of the ICT organization of the Dutch universities SURF: Clearing the Gate. This initiative is aimed at funding organizations such as the Dutch research council NWO. It calls upon them to develop a preference for open access publications for the research they fund. They should give priority to publications in high quality open access journals as a condition for funding. SURF is convinced that once this priority is installed, we will witness a strong growth in the number of available open access journals of a high to very high quality. The presentative of NWO joined this initiative and made clear that his organization already supports new open access journals in the social sciences and humanities. This Spring, NWO will publish a Call aimed at the other disciplines. NWO also supports the OAPEN initiative for open access books in the humanities. An important motivation for the organization is financial: “we do not want to pay twice for the same research”. For evaluators and scientometricians, this development is an interesting challenge as well. How to evaluate open access activities in research?

Note: My Dutch language report of the EU Open Access workshop meeting was published in the journal Onderzoek Nederland, nr. 277, 7 May 2011, p. 8.

My presentation at the EU workshop is available here.

Open Access in European Science Policy

http://prezi.com/_bfvsoqb7pvp/acumen-academic-careers-understood-through-measurement-and-norms/

Attended an interesting workshop today on Coordinating Open Access in European Science Policy. We presented our ACUMEN project there, making the connection with the current Open Access debates. Here is the full presentation.

Can metrics support open access monograph publishing?

Academic publishers are developing new ideas about open access in response to the economic crisis in scholarly publishing. To be sure, the open access movement was not started by publishers. It emerged from within the academic community in response to the serials crisis in university libraries and the rising prices of journals. Since 1995, open acccess has moved from the periphery to the centre of academia. The Berlin Declaration (2003) was an important moment that signaled that it was supported by most representatives of the scientific and scholarly establishment. Funding agencies are also moving towards open access as the prefered model as is the European Union, albeit slowly. This raises the question of how citation analysis and performance metrics can be developed that fit with this process. Metrics may either be used to monitor the growth and impact of open access publications. Metrics might also be used by researchers whose performance is evaluated to make the case that they contribute to the wider dissemination of their research.

A specific instance of open access publishing is the development of open access monographs. In this area, academic publishers are taking the lead, not coincidentally from smaller countries, prominent among them Amsterdam University Press. They are edged on by the rapid decline of the publication of printed scholarly books. This is partly caused by a shift from the production of monographs to articles in international peer reviewed journals in a number of fields in the social sciences and humanities. Also, book publishing is an expensive affair. If the sales numbers keep dropping it will soon become virtually impossible to publish scholarly monographs unless they also serve a much larger, non-specialist, audience. The problem is, of course, that book-length arguments are still important to synthesize research findings. This is not limited to the humanities or social sciences, but it is especially urgent in these fields. Can open access be (part of) the answer to this threat?

This was the topic of a conference organized by OAPEN on 25 February this year in Berlin. In my contribution to this conference, I have tried to sketch a possible research agenda to develop appropriate metrics for open access books. The issue of this type of metrics is a combination of four different measurement challenges. Each one of them is already a considerable problem: the measurement of open access (until now mostly developed for journals); the bibliometrics of the humanities and social sciences (plagued by the problem of inadequate source coverage by the Web of Science); webometrics of the use of information sources on the Web (confronted with enormous amounts of noise in the data); and the problem of measuring societal impact of research (often denoted with ‘societal quality’ – a problem of measuring widely diverging interactions).

If open access book metrics would simply be the sum of these four, it would probably be an impossible challenge to develop such a metrics. A research strategy would therefore need to focus on finding generic solutions that may be valid for more than one of these dimensions.

Currently, we simply do not have the tools to properly measure academic open acccess books. But we may use existing work as building blocks. My proposal is to follow a threefold strategy that works in parallel on three different tasks.

The first is to develop a model which describes how scholarly monographs are actually functioning in processes of the creation and use of research and scholarly knowledge. We already have some of these models focused on the economics of book publishing and these could be refined in terms of the information flows that the books are both an expression of and an element in.

The second task would be to work on data production and curation to ensure that the publication process captures the relevant data and that all parties agree on the crucial standards and technical norms needed. An prominent initiative is the COUNTER consortium, that aims to develop “an agreed international set of standards and protocols governing the recording and exchange of online usage data”. Our colleagues in Madrid have moreover developed a ranking method of Web based repositories that may be useful to build on in a metrics of open access books.

The third task is the creation of meaningful indicators of open access monographs. It may be useful to distinguish different dimensions: monitoring the creation or production of open access book sources; measuring their use by both scholars and lay audiences; and lastly their impact, implications, or influence on the creation and use of scientific and scholarly knowledge.

Anyone interested in joining forces in this endeavour?