New CWTS blog

Dear readers,

It is with great pleasure that we announce a new platform for blog posts emanating from our institute: the CWTS blog.

Screen Shot 2015-11-04 at 10.06.51

How important is the number of citations to my scientific work really? How is evaluation influencing knowledge production? Should my organisation support the DORA declaration? When does it (not) make sense to use the h-index? What does a competitive yet conscientious career system look like? What is the relation between scientific and social impact of research? How can we value diversity in scholarship?

The CWTS blog brings together ideas, commentary, and (book) reviews about the latest developments in scientometrics, research evaluation, and research management. It is written for those interested in bibliometric and scientometric indicators and tools, implications of monitoring, measuring, and managing research, and the potential of quantitative and qualitative methods for understanding the dynamics of scientific research.

This a moderated blog with a small editorial team consisting of Sarah de RijckeLudo Waltman, and Paul Wouters. The blog posts are written by researchers affiliated to CWTS.

In the meantime, this current Citation Culture blog will be discontinued. Thank you all very much for your dedicated readership. We hope you will enjoy reading the new blog!

You can subscribe to the mailinglist or rss feed at

The Facebook-ization of academic reputation?

Guest blog post by Alex Rushforth

The Facebook-ization of academic reputation? ResearchGate, and Everyday neoliberalism

How do we explain the endurance of neoliberal modes of government following the 2008 financial crisis, which could surely have been its death-knoll? This is the question of a long, brilliant, book by historian of science and economics Philip Mirowski, called ‘Never let a serious crisis go to waste’. Mirowski states that explanations of the crisis to date have accounted for only part of the answer. Part of the persistence of neo-liberal ideals of personhood and markets comes not just directly from ‘the government’ or particular policies, but is a result of very mundane practices and technologies which surround us in our everyday lives.

I think this book can tell us a lot about new ways in which our lives as academics are increasingly being governed. Consider web platforms like ResearchGate and following Mirowski, these academic professional networking sites might be understood as technologies of ‘everyday neoliberalism’. These websites share a number of resemblances with social networking sites like Facebook – which Mirowski takes as an exemplar par excellence of this phenomenon. He argues Facebook teaches its users to become ‘entrepreneurs of themselves’, by fragmenting the self and reducing it to something transient (ideals emanating from the writings of Hayek and Friedman), to be actively and promiscuously re-drawn out of various click-enabled associations (accumulated in indicators like numbers of ‘likes’, ‘friends’, comments) (Mirowski, 2013, 92).

Let us briefly consider what kind of academic and ResearchGate encourages and teaches us to become. Part of the seductiveness of these technologies for academics, I suspect, is that we already compete within reputational work organisations (c.f. Whitley, 2000), where self-promotion has always been part-and-parcel of producing new knowledge. However, such platforms also intensify and reinforce dominant ideas and practices for evaluating research and researchers, which – with the help of Mirowski’s text – appear to be premised on neoliberal doctrines. Certainly the websites build on the idea that the individual (as author) is the central locus of knowledge production. Yet what is distinctly neoliberal perhaps is how the individual – through the architecture and design of the websites – experiences their field of knowledge production as a ‘marketplace of ideas’ (on the neo-liberal roots of this idea, see Mirowski, 2011).

This is achieved through ‘dashboards’ that display a smorgasbord of numerical indicators. When you upload your work, the interface generates the Impact Factor of journals you have published in and various other algorithmically-generated scores (ResearchGate score anyone?). There are also social networking elements like ‘contacts’, enabling you to follow and be followed by other users of the platform (your ‘peers’). This in turn produces a count of how well ‘networked’ you are. In short, checking one’s scores, contacts, downloads, views, and so on is supposed to give an impression of an individual user’s market standing, especially as one can compare these with scores of other users. Regular email notifications provide reminders to continue internalizing these demands and to report back regularly to the system. These scores and notices are not final judgments but a record of accomplishments so far, motivating the user to carry on with the determination to do better. Given the aura of ‘objectivity’ and ‘market knows best’ mantra these indicators present to us, any ‘failings’ are the responsibility of the individual. Felt anger is to be turned back inward on the self, rather than outwards on the social practices and ideas through which such ‘truths’ are constituted. A marketplace of ideas indeed.

Like Facebook, what these academic professional networking sites do seems largely unremarkable and uncontroversial, forming part of background infrastructures which simply nestle into our everyday research practices. One of their fascinating features is to promulgate a mode of power that is not directed to us ‘from above’ – no manager or formal audit exercise is coercing researchers into signing-up. We are able to join and leave of our own volition (many academics don’t even have accounts). Yet these websites should be understood as component parts of a wider ‘assemblage’ of metrics and evaluation techniques with which academics currently juggle, which in turn generate certain kinds of tyrannies (see Burrows, 2012).

Mirowski’s book provides a compelling set of provocations for digital scholars, sociologists of science, science studies, higher education scholars and others to work with. Many studies have been produced documenting reforms to the university which have bared various hallmarks of neoliberal political philosophical doctrines (think audits, university rankings, temporary labour contracts, competitive funding schemes and the like). Yet these latter techniques may only be the tip of the iceberg: Mirowski has given us cause to think more imaginatively about how ‘everyday’ or ‘folk’ neoliberal ideas and practices become embedded in our academic lives through quite mundane infrastructures, the effects of which we have barely begun to recognise, let alone understand.


Burrows, R. 2012. Living with the h-index? Metric assemblages in the contemporary academy. Sociological Review, 60, 355-372.

Mirowski, P. 2011. Science-mart : privatizing American science, Cambridge, Mass. ; London, Harvard University Press.

Mirowski, P. 2013. Never let a serious crisis go to waste : how neoliberalism survived the financial meltdown, New York, Verso.

Whitley, R. 2000. The intellectual and social organization of the sciences, Oxford England ; New York, Oxford University Press.



CWTS in new European consortium

Good news came our way yesterday! CWTS will be partner in a new project funded by the Swedisch Riksbankens Jubileumsfond: Knowledge in science and policy. Creating an evidence base for converging modes of governance in policy and science (KNOWSCIENCE). The project is coordinated by Merle Jacob (Lund University, Sweden). Other partners in the consortium are Dietmar Braun (Lausanne University, Switzerland), Tomas Hellström (Department of Business Administration, Lund University), Niilo Kauppi (CNRS, Strasbourg, France), Duncan Thomas & Maria Nedeva (Manchester Institute of Innovation Research, Manchester Business School, UK), Rikard Stankiewitz (Lund University), and Sarah de Rijcke & Paul Wouters (CWTS).

KNOWSCIENCE focuses on deepening our understanding of the interplay between policy instruments intended to govern the structural organization of higher education and research (HER) and the informal rules and processes that organisations have developed for ensuring the validity and quality of the knowledge they produce. KNOWSCIENCE refers to this as the interplay between structural and epistemic governance, and argue that an understanding of this relationship is necessary for building sustainable knowledge producing arrangements and institutions and securing society’s long-term knowledge provision.

The main research question guiding the project is ‘how do policy and the science systems co-produce the conditions for sustainable knowledge provision?’ Specifically we ask:

(a) How are HER policy steering mechanisms enabled, disabled and transformed throughout the HER sector via the academic social system?

(b) What are the most significant unintended consequences of HER policy on the HER system? and

(c) What types of policy frameworks would be required to meet these challenges?

The announcement on the RJ website can be found via this link.

Indicator-considerations eclipse other judgments on the shop-floor | Keynote Sarah de Rijcke ESA Prague, 26 August 2015

This invited lecture at the ESA conference in Prague drew on insights from the Leiden Manifesto and from two recent research projects at our institute in the Evaluation Practices in Context research group. These research projects show how indicators influence knowledge production in the life sciences and social sciences, and how in- and exclusion mechanisms get built into the scientific system through certain uses of evaluative metrics. Our findings point to a rather self-referential focus on metrics and a lack of space for responsible, relevant research in the scientific practices under study. On the basis of these findings I argued in the talk that we need an alternative moral discourse in research assessment, centered around the need to address growing inequalities in the science system. Secondly, the talk considered the most pertinent issues for the community of sociologists from the Leiden Manifesto for research metrics (Hicks, Wouters, Waltman, De Rijcke & Rafols, Nature, 23 April 2015).

See also:

Rushforth & De Rijcke (2015). Accounting for Impact? The Journal Impact Factor and the making of biomedical research in the NetherlandsMinerva, 53(2), 117-139.

De Rijcke, S. & Rushforth, A.D. (2015). To intervene, or not to intervene, is that the question? On the role of scientometrics in research evaluation. Journal of the Association for Information Science and Technology, 66 (9), 1954-1958.

Hicks, D., Wouters, P.F., Rafols, I., De Rijcke, S. & Waltman, L. (2015). The Leiden Manifesto for Research Metrics. Nature, 23 April 2015.

Hammarfelt & De Rijcke (2015). Accountability in Context: Effects of research evaluation systems on publication practices, disciplinary norms, and individual working routines in the faculty of Arts at Uppsala UniversityResearch Evaluation, 24(1), 63-77.

Leiden Manifesto for research metrics published in Nature

We’re happy to announce the publication of ten principles to guide the use of metrics in research evaluation – a collaboration between Diana Hicks (Georgia Tech), Ismael Rafols (Ingenio/SPRU), Paul Wouters, Sarah de Rijcke and Ludo Waltman (CWTS). The principles were formulated on the basis of input from the scientometric community during a special session on the development of good practices for metrics use at the STI/ENID conference in Leiden (September 2014).


CWTS part of H2020 COST Action to stimulate integrity and responsible research

Good news came our way recently! Thed van Leeuwen, Paul Wouters and myself will be part of an EC-funded H2020 COST Action on Promoting Integrity as an Integral Dimension of Excellence in Research (PRINTEGER). Main applicants Hub Zwart and Willem Halffman (Radboud University Nijmegen) brought together highly skilled partners for this network from the Free University Brussels, the University of Tartu (Estonia), Oslo and Akershus University College, Leiden University, and the Universities of Bonn, Bristol, and Trento.

The primary goal of the COST Action is to encourage a research culture that treats integrity as an integral part of doing research, instead of an externally driven steering mechanism. Our starting point: in order to stimulate integrity and responsible research, new forms of governance are needed that are firmly grounded in and informed by research practice.

Concretely, the work entailed in the project will consist of A) a systematic review of integrity cultures and practices; B) an analysis and assessment of current challenges, pressures, and opportunities for research integrity in a demanding and rapidly changing research system; and C) the development and testing of tools and policy recommendations enabling key players to effectively address issues of integrity, specifically directed at science policy makers, research managers and future researchers.

CWTS will contribute to the network with

  • A bibliometric analysis of ‘traces of fraud’ (e.g. retracted articles, manipulative editorials, non-existent authors and papers, fake journals, bogus conferences, non-existent universities), against the background of general shifts in publication patterns, such as changing co-authoring practices, instruments as authors, or the rise of hyper-productive authors;
  • Two in-depth cases studies of research misconduct, not the evident or spectacular, but more particularly reflecting dilemmas and conflicts that occur in grey areas. Every partner will provide two cases; ours will most likely focus on cases of questionable integrity of journal editors (for example cases of impact factor manipulation);
  • Act as task leader on formulation of Advice for research support organisations, including on IT tools. This task will draw conclusions from the research on the operation of the research system, specifically publication infrastructures such as journals, libraries, or data repositories;
  • Like all other partners in the network, we will set up small local advisory panels consisting of five to ten key stakeholders of the project: research policy makers, research leaders or managers, research support organisations, and early career scientists. These panels will meet for a scoping consultation at the start of the projects, for a halfway consultation to discuss intermediate results and further choices to be made, and for a near-end consultation to test the pertinence of tools and advice at a point where we can still make changes to accommodate for stakeholder input.

Knowledge, control and capitalism: what epistemic capitalism makes us see

Guest blog post by Thomas Franssen

On February 5th Max Fochler (University of Vienna) gave a talk during an extended EPIC research group seminar at the CWTS in Leiden. Fochler posed a crucial and critical question regarding knowledge production in the 21th century; can we understand contemporary practices of knowledge production in and outside academia as practices of epistemic capitalism? With this term, defined as ‘the accumulation of capital through the act of doing research’, Max wanted to address the regimes of worth that play a crucial role in life science research in Austria.

Max was interested in exploring the concept of capitalism as it denotes both forms of ascribing worth or value to something (in this case to knowledge and doing research), and the sets of practices in which these forms of worth are embedded. In this way it allows one to talk about which registers or regimes of value are visible as well as the institutional context in which these forms of worth ‘count’ for something.

Using research on the life sciences (partly done with Ulrike Felt and Ruth Müller) Max compared the regimes of value found in biographical interviews with postdocs working in Austrian academia to those of founders, managers and employees of research-active small biotech companies in Austria.

Results showed that the postdocs in their study are preoccupied with their own future employability, and that they assess their own worth in terms of the capital that they can accumulate. This capital consists of publications, impact factors, citations and grant-money. What is especially critical in this respect is that potential sites of work, social relations with others, and choices for particular research topics or model organisms are scrutinized in relation to the effect they might have on the accumulation of capital. Importantly, also for research policy and higher education studies, this is the only strategy that this sample of postdocs sees as viable. They do not see other regimes of valuation available for them. As such they either comply to the rules of the game or opt out of the academic system entirely.

In biotech companies the situation is very different. The accumulation of epistemic capital plays a smaller role in the biographies of those working for biotech companies. The main difference, Max observed, is that failure and success are attributed to companies rather than individuals. The intense competition and focus on the individual as experienced by postdocs in the life sciences is less intense in biotech. As such, the essence of working in biotech is not the accumulation of capital, but the development of the company. Capital is not an end in itself but used strategically when possible.

Thinking through epistemic capitalism with biodiversity

Esther Turnhout (Wageningen University) was invited by Sarah to comment on Max Fochler’s talk. Turnhout’s research focuses on information and accountability infrastructures in forest certification and auditing in global value chains. She started her response by asking whether the concept of epistemic capitalism made her look at her own case stuy materials differently and if so how? Not to interrogate the concept and test it empirically but rather to make clear what it highlights and what it affords.

Her criticism of the term came down to two aspects, which she explained using the case of biodiversity. Most importantly, the concept of epistemic capitalism ties the development of knowledge to the accumulation of capital directly and it has the tendency to reduce everything it captures to one single mechanism or logic.

To make her case, Esther traced the knowledge making practices in biodiversity research historically. She did so by focusing on the rise of so-called ecosystem services. Within ecosystem services biodiversity knowledge has become mainly utilitarian, and biodiversity itself an object that presents economic value because it has not yet been destroyed. Think for example of forest carbon, which represents a value on the carbon market as long as it is locked in the forest itself.

So here, in the commodification of biodiversity, knowledge and capital are again closely related. This, however, is not the main argument that Esther took from this example. Rather, she argued that in many ways ecosystem services are very similar to the history of biodiversity knowledge. In all cases, the knowledge produced must be rendered technical, it is assumed to be linear and it privileges scientific expertise. More importantly, there is a preoccupation with ‘complete knowledge’, which is seen as needed for effective conservation. Also, this type of knowledge is increasingly used for managerial concerns to measure success or effectiveness of policy.

As such, disconnected from capitalist or economic concerns, in biodiversity knowledge three logics come together: a technocratic logic, a managerial logic, and a logic of control. For her case, a focus on epistemic capitalism and the accumulation of capital does not work so well. The issue of a technocratic ideal of total control would disappear from view if ecosystems services are only regarded as a commodification of nature. It is the issue of control, which can be understood from a range of logics (technocratic, managerial even aesthetic), that currently prevents urgently needed action. This is because there is an experienced lack of ‘total information’, a total which – seen from technocratic and managerial logics – is needed to act. According to Turnhout it is this utopian ideal of ‘technocratic control through complete information’ that should be criticised much more strongly.

%d bloggers like this: