Modelling the peer review system a la SIMCITY

http://www.socialsciences.leiden.edu/cwts/sti-conference2010/

At the STI conference 2010 my colleagues Andrea Scharnhorst, Krzysztof Suchecki from the Virtual Knowledge Studio and I presented our work in progress on modeling the peer review system. The basic idea is simple: is it possible to model the peer review system as if it were a computer game such as Simcity? We followed a strategy where we try to make the model as simple and stupid as possible. So, iniitally we are not trying to mimic reality, but to set up an extremely simplified model of how peer review works in science and academia. Our model consists of two populations: researchers and journals. The researchers have two different roles: they are authors of scientific papers and they are reviewers who judge the quality of scientific papers written by other researchers. Each researcher has her own specific behaviour and the same holds for the journals. The trick of the model is that we incorporated a simulation of quality control, using multi-dimensional vectors. This is extracted from what we know how peer review works. Bascially, reviewers are comparing what they perceive of the work in different dimensions (such as the quality of writing, the images, the statistical reliability, how interesting the quesions are, etc.) with what they perceive as the required quality. We assume that this expected quality relates to the quality of the work that the researcher produces herself. The project is in an early stage, and we are now in the process of writing it up for a proper first publication, mainly on the methodology. At the conference we presented the following poster, that contains more details (I posted it on my Facebook account since this blog software system is apparently not able to process images unless they are very small):

http://www.facebook.com/home.php?#!/photo.php?pid=5014709&fbid=440206878395&id=506988395.

 

About these ads

One Response to “Modelling the peer review system a la SIMCITY”

  1. Anne Beaulieu Says:

    Have you seen this, speaking of understanding peer-review models?

    http://www.plosone.org/article/info%3Adoi%2F10.1371%2Fjournal.pone.0008092


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: