What kind of societal relevance do we want?

The question about the societal relevance and economic impact of research is increasingly put forward in peer review of research projects, annual appraisal interviews, and institutional research assessment exercises. This is part of a more generic trend in accountability practices in a variety of societal sectors and in the way strategic intelligence is managed in business processes. An important task for CWTS will be to contribute actively to the build-up of conscientious criteria for this relatively new research assessment module. CWTS recently hired dr. Ingeborg Meijer – previously senior consultant at Technopolis and scientific officer at the Advisory Counsil on Health Research – to take on this task. Ingeborg presented her plans at last week’s CWTS research seminar.

The increasing focus on the societal impact of research has created a serious problem for both researchers and evaluators because these wider impacts of the outcomes of research (different from the more narrow research output in the form of publications) are very difficult to prove and evaluate. This is mainly caused by the complex nature of the interactions between academia, industry, and the public sector. There is no straightforward way out of this problem. Assessing the social, economic, cultural and ecological impact of scientific research is not simply a matter of developing performance indicators for ‘societally relevant’ research activity and an accompanying technological infrastructure for data collection. The methods and techniques for evaluating societal relevance will themselves also affect how ‘societal impact’ is defined and operationalized.

Using indicators and methods for research assessment is not merely a descriptive but also a prescriptive practice. Bear in mind some of the perverse effects of quantitative performance indicators for scientific impact: in some fields the citation culture seems to have resulted in an unhealthy interest in uni-dimensional output measures such as the number of articles published in high-impact journals and the number of times these articles are cited. If we take seriously that research assessment is a social technology we should also acknowledge these undesirable effects. It may indeed be beneficial for researchers if there is more balance in the types of activitities they will be held accountable for. As Ingeborg pointed out, making visible the ‘societally relevant’ work researchers are already doing (by collecting data on the web or by asking researchers to list activities, for instance) is a promising start. In addition, and considering the performative effects of indicators, policy makers, researchers and evaluation officers should also develop an overarching vision on the kinds of work they deem crucial and ‘socially relevant’. The activities that are currently being mapped out are undertaken within (and will therefore reflect) the parameters of the present evaluation system, which lean towards counting international peer-reviewed articles. Perhaps researchers should be encouraged to develop a much more variegated set of activities than they are at present receiving credit for.

Journal ranking biased against interdisciplinary research

The widespread use of rankings of journals in research institutes and universities creates a disadvantage for interdisciplinary research in assessment exercises such as the British Research Excellence Framework. This is the conclusion of a paper presented at the 2011 Annual Conference of the Society for the Social Studies of Science in Cleveland (US) by Ismael Rafols (SPRU, Sussex University), Loet Leydesdorff (University of Amsterdam) and Alice O’Hare, Paul Nightingale and Andy Stirling (all SPRU, Sussex University). The study is the first quantitative proof that researchers working at the boundaries between different research fields may be disadvantaged compared with monodisciplinary colleagues. The study argues that citation analysis, if properly applied, is a better measurement instrument than a ranked journal list.

The study is quite relevant for research management at universities and research institutes. Journal lists have become a very popular management tool. In a lot of departments, researchers are obliged to publish in a limited set of journals. Some departments, for example in economics, have even been reorganized on the basis of having published in such a list. The way these lists have been composed does vary. Sometimes a group of experts decides whether a journal belongs to the list, sometimes the Journal Impact Factor published by ISI/Thomson Reuters is the determining factor.

The study by Rafols et al. has analyzed one such list: the ranked journal list used by the British Association of Business Schools. This list is based on a mix of citation statistics and peer review. It ranks scholarly journals in business and management studies in five categories. “Modest standard journals” are category 1, “world elite journals” are category 4*. This scheme reflects the experience researchers have with the Research Assessment Exercise categories. The ranked journal list is meant to be used widely for a variety of management goals. It is used as an advice for researchers about the best venue for their manuscripts. Libraries are supposed to use it in their acquisition policies. And last but not least, it is used in research assessments and personnel evaluations. Although the actual use of the list is an interesting research topic in itself, we can safely assume that it has had a serious impact on the researchers in the British business schools community.

The study shows first of all that the position of a journal in the ranked list correlates negatively with the extent of interdisciplinarity of the journal. In other words, the higher the ranking, the more narrow its disciplinary focus. (The study has used a number of indicators for interdisciplinarity by which different aspects of what it means to be interdisciplinary have been captured.) Rewarding researchers to publish first of all in the ranked journal list may therefore discourage interdisciplinary work.

The study confirms this effect by comparing business and management studies to innovation studies. Both fields are subjected to the same evaluation regime in the Research Excellence Framework. Intellectually, they are very close. However, they differ markedly with respect to their interdisciplinary nature. Researchers in business schools have a more traditional publishing behaviour than their innovation studies colleagues. The research units in innnovation studies are consistently more interdisciplinary than the business and management schools.

Of course, publication behaviour is shaped by a variety of influences. Peer review may be biased against interdisciplinary work because it is more difficult to assess its quality. Many top journals are not eager to publish interdisciplinary work. This study is the first to show convincingly that these already existing biases tend to be made even stronger by the use of ranked journal lists as a tool in research management. The study confirms this effect by comparing the performance based on the ranked journal list with a citation analysis. In the latter, the innovation studies research is not punished by its more interdisciplinary character which does happen in an assessment on the basis of the journal list. The paper concludes with a discussion of the negative implications in terms of funding and acquiring resources for research groups working at the boundaries of different fields

The paper will be published in a forthcoming issue of Research Policy and has been awarded the best paper at the Atlanta Conference on Science and Innovation Policy in September 2011.

Reference: Ismael Rafols, Loet Leydesdorff, Alice O’Hare, Paul Nightingale, & Andy Stirling, “How journal rankings can suppress interdisciplinary research. A comparison between innovation studies and business & management,” Paper presented at the Annual Meeting of the Society for the Social Studies of Science (4S), Cleveland, OH, Nov. 2011; available at http://arxiv.org/abs/1105.1227 .

Harvard no longer number 1 in ranking

Recently, the new Times Higher Education World University Rankings 2011-2012 saw the light. The ranking revealed that Harvard University is no longer number one on the list. Incidentally, the differences with Caltech – now highest – are minimal. The main reason for Caltech’s rise are the extra revenues it drew out of industry. Caltech’s income increased by 16%, thereby outclassing most other universities. Harvard scored a bit better when it comes to the educational environment. Other universities also rose on the list as a result of a successful campaign to obtain (more) external financing. The London School of Economics, for example, moved from 86 to 47. The top of the ranking did not change that drastically though. Rich US-based universities still dominate the list. 7 out of ten universities highest on the list, and one third of the top 200, are located in the US.

This illustrates the THE ranking’s sensitivity to slight differences between indicators that, taken together, shape the order of the ranking. The ranking is based on a mix of many different indicators. There is no standardized way to combine these indicators, and therefore there inevitably is a certain arbitrariness to the process. In addition, the THE ranking is partly based on results of a global survey. This survey invites researchers and professors to assess the reputation of universities. One of the unwanted effects of this method is that well-known universities are more likely to be positively assessed than less popular universities. Highly visible forms of maltreatment and scandals may also influence survey results.

This year, the ranking’s sensitivity to the ways in which different indicators are combined is aptly illustrated by the position of the Dutch universities. The Netherlands are at number 3, with 12 universities in the top 200 and 4 in the first 100 of the world. Given the size of the country, this is a remarkable achievement. The result is partly caused by a strong international orientation of the Dutch universities, and partly by previous investments in research and education. But just as important is the weight given to the performances of the social sciences and humanities in a number of indicators. Compared to last year, the total performance of Dutch universities most likely did not increase that much. A more likely explanation is that the profile of activities and impact are better covered by the THE ranking.

The latest THE ranking does make clear that size is not the most important determinant in positioning universities. Small specialized universities can end up quite high on the list.

Perspectives on computer simulation and data visualization

When it comes to critical analysis of the role of computers, data visualization, simulations and modeling in the sciences, much can be learned from humanities scholars. I’m currently teaching a course on the role of computer-generated images in contemporary science and visual culture at Utrecht University. Yesterday I learned that the New Media department hosts two very interesting events. Today, Tuesday October 18, there’s a workshop on software applications as active agents in shaping knowledge. The two keynote speakers are Dr Eckhart Arnold (University of Stuttgart), expert in the field of simulation technologies, and Dr Bernhard Rieder (University of Amsterdam), who researches how computers and software organize knowledge.

A week later, on October 25, Setup will host an event on data visualization at the Wolff Cinema movie theatre in Utrecht. Some of the most striking recent data visualization projects will be displayed on screen, and the following questions will be addressed: what makes data visualizations so appealing? Do they bring across the same message as the ‘raw’ data they originate from? Ann-Sophie Lehmann (associate professor New Media en Art History, UU) will discuss the visualizations and will throw light on some of the effects they have on viewers. One question that came to my mind is what this particular context (a movie theater) does to the (reception of) the visualizations, compared to a web-based interaction on a laptop or PC, for instance.

Understanding Academic Careers

On November 16, 2011, the Rathenau Institute and the VU University Amsterdam organize a symposium on Dynamics of Academic Leadership. The symposium addresses the conditions that are necessary for high levelperformance and creativity in research, and the implications for researchmanagement and policy. Paul is one of the invited speakers. He will discuss some of the programmatic aspects and preliminary results of a large European FP-7 project: Academic Careers Understood through Measurement and Norms (ACUMEN). ACUMEN is aimed at understanding theways in which researchers are evaluated by their peers and institutions,and at assessing how the science system can be improved and enhanced. The project is a cooperation among several European research institutes, with Paul as the principalinvestigator and CWTS’s Clifford Tatum as project manager.

Science mapping: do we know what we visualize?

A recent landmark in the field of science mapping is Katy Börner’s Atlas of Science: Visualizing What We Know (MIT Press, 2010). The atlas recently won the ASIS&T Best Information Science Book Award 2011.The kinds of maps covered by the atlas range from historical timelines, network diagrams and citation networks revealing rises in patent citations, to geographic maps, taxonomic hierarchies and maps of relative sizes and connectedness of scientific fields.

The advent of science mapping depends to a large extent on digitized indices of scholarly activity such as the Science Citation Index, and on advances in network analysis and visualization techniques. Bibliometric maps of scholarly activity are mostly based on bibliographic coupling, co-citation analyses or maps of keywords based on a co-occurrence network. The visualizations that are created are transformations of quantified data into visual form. The avalanche of bibliometric data incorporated in massive databases demand new visualization tools and – crucially – the skills to understand and engage with these new kinds of visualizations.

Most bibliometric mapping endeavors radiate an ambition on the part of the scientist(s) producing these maps to be synthetic, comprehensive and definitive. Börner’s Atlas of Science, for instance, is said to chart “the trajectory from scientific concept to published results,” revealing “the landscape of what we know.” However, maps are not a direct reflection of reality, all sorts of decisions are taken to process the data before they can be presented. While this may seem a matter ‘of course’, it does have consequences for the interpretation and  use of these maps.

For example, what often gets glossed over in these endeavors is that visualizations of scientific developments also prescribe how these developments should be known in the first place. Science maps are produced by particular statistical algorithms that might have been chosen otherwise, calculations performed on large amounts of ‘raw’ data , and for this reason they are not simply ‘statistical information presented visually’. The choice for a particular kind of visualization is often connected to the specificities and meaning of the underlying dataset and the software used to process the data. Several software packages have been specifically designed for this purpose (the VOSViewer supported by CWTS being one of them). These packages prescribe how the data should be handled. Different choices in selection and processing of the data will lead to sometimes strikingly different maps. Therefore, we will increasingly need systematic experiments and studies with different forms of visual presentation (Tufte, 2006).

At the same time, a number of interfaces are built into the mapping process, where an encounter takes place with a user who approaches these visualizations as evidence. But how do these users actually behave? To our knowledge hardly any systematic research is done on how users (bibliometricians, computer scientists, institute directors, policy makers and their staff, etc.) engage with these visualizations, and which skills and strategies are needed to engage with them. A critical scrutiny is needed of the degree of ‘visual literacy’ (Pauwels, 2008) demanded of users who want to critically work with and examine these visualizations. The visualizations technical or formal choices that determine what can be visualized and what will remain hidden. Furthermore, they are also shaped by the broader cultural and historical context in which they are produced.

Unfortunately there is a tendency to downplay the visuality of science maps, in favor of the integrity of the underlying data and the sophistication of transformation algorithms. However, visualizations are “becoming increasingly dependent upon technology, while technology is increasingly becoming imaging and visualization technology” (Pauwels 2008, 83). We expect that this interconnection between data selection, data processing and data visualization will become much stronger in the near future. These connections should therefore be systematically analyzed, while the field develops and experiments with different forms of visual representation.

Science mapping projects do not simply measure and describe scientific developments – they also have a normative potential. the director of a research institute wants to map the institute’s research landscape in terms of research topics and possible applications, and wants to see how the landscape develops over the next five years. This kind of mapping project, like any other description of reality, is not only descriptive but also performative. In other words, the map that gets created in response to this director’s question also shapes the reality it attempts to represent. One possible consequence of this hypothetical mapping project could be that the director decides on the basis of this visual analysis to focus more on certain underdeveloped research strands, at the expense of or in addition to others. The map that was meant to chart the terrain now becomes embedded in management decision processes. As a result, it plays an active part in a shift in the institute’s research agenda, an agenda that will be mapped in five years’ time with the same analytical means that were originally merely intended to describe the landscape.

A comparable example can actually be found in Börner’s book: a map that shows all National Institute of Health (NIH) grant awards from a single funding year., giving access to a database and web-based interface. The clusters on the map correspond to broader scientific topics covered in the grants, while the dots correspond with individual grants clustered together by a shared topical focus.

Here, too, it would be informative to analyze the potential role these maps play as policy instruments (for instance, in accountability studies). This type of analysis will be all the more urgent when bibliometric maps are increasingly used for the purposes of research evaluation. The maps created on the basis of bibliometric data do not simply ‘visualize what we know’. They actively shape bibliometric knowledge production, use and dissemination in ways that require careful scrutiny.

Still using the Hirsch index? Don’t!

“My research: > 185 papers, h-index 40.” A random quote from a curriculum vitae in the World Wide Web. Sometimes, researchers love their Hirsch index, better known as the h-index. But what does the measure actually mean? Is it a reliable indicator of scientific impact?

Our colleagues Ludo Waltman and Nees Jan van Eck have studied the mathematical and statistical properties of the h-index. Their conclusion: the h-index can produce inconsistent results. For this reason, it is actually not the reliable measure of scientific impact that most users think it is. As a leading scientometric institute, we have therefore published the advice to all universities, funders, and academies of science to abandon the use of the h-index as a measure of the overall scientific impact of researchers or research groups. There are better alternatives. The paper by Waltman and Van Eck is now available as a preprint and will soon be published by the Journal of the American Society for Information Science and Technology JASIST.

The h-index is a measure of a combination of productivity and citation impact. It is calculated by ordering the number of publications by a particular researcher on the basis of the total number of citations they have received. For example, someone who has an h-index of 40 has published at least 40 articles that have each been cited at least 40 times. Moreover, the remaining articles have not been cited more than 40 times each. The higher the h-index the better.

The h-index was proposed by physicist Jorge Hirsch in 2005. It was an immediate hit. Nowadays, there are about 40 variants of the h-index. About one quarter of all articles published in the main scientometric journals have cited Hirsch’ article in which he describes the h-index. Even more important has been the response by scientific researchers using the h-index. The h-index has many fans, especially in the fields that exchange many citations, such as the biomedical sciences. The h-index is almost irrresistable because it seems to enable a simple comparison of the scientific impact of different researchers. Many institutions have been seduced by the siren call of the h-index. For example, the Royal Netherlands Academy of Arts and Sciences (KNAW) in the Netherlands inquires about the value of the h-index in its recent forms for new members. Individual researchers can look up their h-index based on Google Scholar documents via Harzing’s website publish or perish. Both economists and computer scientists have produced a ranking of their field based on the h-index.

Our colleagues Waltman and Van Eck have now shown that the h-index has some fatal shortcomings. For example, if two researchers with a different h-index co-author a paper together, it may lead to a reversal of their position in an h-index based ranking. The same may happen when we compare research groups. Suppose we have two groups and each member of group A has a higher h-index than a paired researcher in group B. We would now expect that the h-index of group A as group is also higher than that of group B. Well, that does not have to be the case. Please note that we are now speaking of a calculation of the h-index based on a complete and reliable record of documents and citations. The problematic nature of the data if one uses Google Scholar as data source is a different matter. So, even when we have complete and accurate data, the h-index may produce inconsistent results. Surely, this is not what one wants using the index for evaluation purposes!

At CWTS, we have therefore drawn the conclusion that the h-index should not be used as measure of scientific impact in the context of research evaluation.

Not much news in new Shanghai rankings

Two weeks before the start of the 2011 academic season, the latest issue of the Academic Ranking of World Universities (ARWU) was published. The response to this ranking in the Netherlands is telling about the importance ascribed to global university rankings. Utrecht University saw its position improved with 2 points and went to number 48. Leiden University went up 15 points and is now second after Utrecht at number 65. All Dutch universities are now listed among the 500 “best universities” in the world. The organization of Dutch universities VSNU was thrilled. This was an “excellent performance”, according to the organization, because “the Shanghai Ranking is in itself already a selection of the five hundred best universities in the world. This means that the Dutch universities belong to the best 3 percent of the total universities in the world (17,000).” In our view, this shows that the VSNU has not really understood the point of this ranking and the rationales behind its construction.

All measurements are preceded by decisions pertaining to the object(s) and focus of measurement. In this categorization process, certain factors will be labeled as relevant and others as less or irrelevant. Decisions will be made pertaining to the parameters of the categories that will be taken into account. These decisions fundamentally shape the subsequent measurements. The ARWU ranking is based on the data of 1,000 universities (the other 16,000 are not taken into account). The ranking strongly favours large universities. Because Nobel Prizes and Field Medals have a strong impact on the total ranking, and other prestigious prizes are not taken into account, the ARWU advantages Anglo-Saxon universities and the universities focused on the exact and medical sciences. From its beginning in 2003, the ARWU ranking is led by US universities, with Harvard as number one. The only non-US universities among the top ten are Oxford and Cambridge.

The way research performance is measured in the Shanghai ranking is also problematic. The number of articles in the journals Nature and Science determine 20 % of the ranking score, but prestigious monodisciplinary journals such as Cell or Physica Acta do not weigh so heavily. Influential humanities researchers are almost invisible in the ranking. Just before the Summer, the European University Association pointed to the disadvantages of the most popular global university rankings. In fact, they only rank the elite of the international university system. Moreover, composite rankings like the Shanghai Ranking merge different aspects of university performance (research, teaching, valorization, social impact) into one number. How this composite number is calculated is rather arbitrary and not always transparent. It is therefore unclear to what extent a change in position has anything to do with change in performance.

For example, it is quite certain that the small improvement of Utrecht University is a fluctuation without any significance. Additionally, even a seemingly robust improvement of the performance of a university can be caused by an individual outlier. According to the website Transfer, the three Dutch universities that saw their position most strongly improved had three individual researchers to thank for this improvement. Radboud University went up thanks to Nobel Prize winner Konstantin Novoselov. Eindhoven’s technical university should send flowers to computer scientist Wil van der Aalst, and Maastricht has risen thanks to behavioural psychologist Gerjo Kok. The fact that individual researchers can have such a strong influence on the position of a university in this ranking may trigger all sorts of perverse behaviour, such as trying to lure staff away from a competing university.

Rankings under Groninger fire

Rafael Wittek, director of the Internuniversity Center for Social Science Theory and Methodology, based at the University of Groningen, recently attacked Dutch university policies at the occasion of the 25^th anniversary of his famous graduate school. One of his targets was “the hype around rankings”. Accredited in 1986, the ICS was the first national social science graduate school in the Netherlands. The school emerged from Dutch networks of PhD students that were funded by the Ministry of Education and Science. According to Wittek, the universities are now trying to get a high score in the global rankings (such as the Times Higher Education ranking, the Shanghai ranking and of course also the Leiden ranking) and he argued that this is a wrongheaded approach. “Rankings as an indicator of quality are a hype. To adopt them is merely a policy reflex.”

I think the sociologist puts his finger on a sore spot in Dutch Science Policy and management. This is particularly true for his critique of the policies around PhD training and the national Graduate Schools. According to Wittek, “The Hague” has been too eager to follow new European guidelines and has promoted the competition, rather than the cooperation, among universities. “In the last couple of years, many national Graduate Schools have been dismantled and new local Graduate Schools have been created in their stead. Dutch universities increasingly claim the results of ‘their’ researchers and give them less possibilities to collaborate with colleagues from other universities”. His remarks will strike a chord with everybody (such as myself) who have been formed in the national research schools in which all or almost all universities worked together. It is indeed a loss that the Dutch ministry discouraged national Graduate Schools and completely switched towards stimulating local ones, although happily a few nation-wide schools are still alive and kicking (such as the Graduate School Science, Technology and Modern Culture).

Still, although his remarks are to the point, I do not think he is completely right. For example, it is simply not true that the Dutch universities would be involved in a ruthless competition with each other. On the contrary, the new trend is the emergence of regional clusters of universities as a new form of intimate collaboration to be able to compete globally with American and Asian universities. Increasing collaboration is moreover the trend in scientific publications, as demonstrated recently by a study of my colleagues at CWTS and by the recent Royal Society Report on scientific networks. The share of the multi-authored, multi-institutional, international publications is still rising, in all fields of research. And their average citation impact is greater than those of single-author or national publications. I don’t think that we should overestimate the power of university boards to limit the scale of scientific collaboration.

Nonetheless, Wittek’s criticism of ranking should certainly be taken very seriously. The sociologist sees a danger in the “policy reflex” for the quality of research and in particular in the areas of high-risk fundamental research. He thinks that researchers who are forced to score high in the rankings will be reluctant to take on big, important questions and will tend to develop a more limited and less risky research agenda. I agree. This is indeed the most important risk of rankings running wild, disconnected from the context of fundamental or applied research. But I think there may be a bit more at play than just policy reflexes. The universities are confronted with an accelerating process of global competition in which new scientific centres are emerging, among others China, India, Brazil, Turkey and Iran. In these countries, researchers tend to have to meet much stricter performance criteria than is usual in the Netherlands. This makes it difficult, perhaps even impossible, for Dutch university boards to ignore this. In the Netherlands this problem is particularly acute since the recent xenophobic hype around immigration in this country is making it already difficult enough to attract talented young researchers from non-European countries. Does this mean that an obsession with rankings is inevitable? I think not. I could imagine a number of alternative, more imaginative strategies to counter this race for the highest position in the rankings.

I do think Wittek is right that recognition by peers is the strongest motivator for researchers. He even thinks that scientists do not need any other stimulus. This last idea may be a bit over the top. But I do think he has a good point. Therefore, rankings can and should be used in direct connection with this peer stimulus. Policies that are only focused on getting higher in the global university rankings indeed do not make much sense. But this does not mean that it makes no sense at all to rank. Rankings can very well be used to get a better understanding of ones strong and weak points (both at the level of individual researchers, groups and institutes, and universities and countries). This can be done while taking into account the specific characteristics of the relevant disciplines. (For different disciplines different databases may be needed to measure the rankings). Ranking in context, that should be possible, shouldn’t it?

Anxiety about quality may hinder open access

Anxiety about the quality of open access journals hinders the further spread of open access publications. This conclusion was cited many times during the recent Co-ordinating workshop on Open Access to Scientific Information, in Brussels on May 4 this year. The workshop was attended by about 70 key players in Open Access and was organized by two EU directorates: Research and Information Society & Media. The critical role of quality control came to the fore in various ways.

Salvatore Mele (CERN), coordinator of the SOAP project presented the results of their study (based on a Web survey) of the attitudes prevailing among researchers with respect to open access. They reveal a remarkable gap between strong support for open access on the one hand and a lack of actual open access publishing on the other hand. 89 % of the researchers say they are in favour of open access publishing. At the same time, only between 8 and 10 % of the articles published are open access. According to the SOAP study, two factors are mainly responsible for this gap: the problem of financing open access publications and the perceived lack of quality of many open access journals. The Journal Impact Factor of journals was also mentioned as a reason not to publish in existing open access journals.

The weight of these factors does vary by field. For example, in chemistry 60 % of the researchers mention financial reasons as barrier to open access, whereas only 16 % of the astronomers see finance as problematic. In astronomy, worries about the quality of journals are mentioned most (by more than half of the astronomers) whereas this is only seen as a problem by about one-fifth of the chemists. This result points, by the way, to the need to develop specific open access policies for different scientific and scholarly fields. For example, in the humanities open access books will be an important issue.

Quality of the journals was also central in a new initiative made public at the workshop by the delegation of the ICT organization of the Dutch universities SURF: Clearing the Gate. This initiative is aimed at funding organizations such as the Dutch research council NWO. It calls upon them to develop a preference for open access publications for the research they fund. They should give priority to publications in high quality open access journals as a condition for funding. SURF is convinced that once this priority is installed, we will witness a strong growth in the number of available open access journals of a high to very high quality. The presentative of NWO joined this initiative and made clear that his organization already supports new open access journals in the social sciences and humanities. This Spring, NWO will publish a Call aimed at the other disciplines. NWO also supports the OAPEN initiative for open access books in the humanities. An important motivation for the organization is financial: “we do not want to pay twice for the same research”. For evaluators and scientometricians, this development is an interesting challenge as well. How to evaluate open access activities in research?

Note: My Dutch language report of the EU Open Access workshop meeting was published in the journal Onderzoek Nederland, nr. 277, 7 May 2011, p. 8.

My presentation at the EU workshop is available here.