Can one person manipulate the position of a whole university in a university ranking such as the Leiden Ranking? The answer is, unfortunately, sometimes yes – provided the processes of quality control in journals do not function properly. A Turkish colleague recently alerted us to the position of Ege University in the most recent Leiden Ranking in the field of mathematics and computer science. This university, not previously known as one of the prestigious Turkish research universities, ranks second with an astonishing value of the PP(top 10%) indicator of almost 21%. In other words, 21% of the mathematics and computer science publications of Ege University belong to the top 10% most frequently cited in their field. This means that Ege University is supposed to have produced twice the amount of highly cited papers as expected. Only Stanford University has performed better.
In mathematics and computer science, Ege university has produced 210 publications (Stanford wrote almost ten times as much). Because this is a relatively small number of publications, the reliability of the ranking position is fairly low, which is indicated by a broad stability interval (an indication of the uncertainty in the measurement). Of the 210 Ege University publications, no less than 65 have been created by one person, a certain Ahmet Yildirim. This is an extremely high productivity in only 4 years in this specialty. Moreover, the Yildirim publications are indeed responsible for the high ranking of Ege University: without them, Ege University would rank around position 300 in this field. This position is therefore probably a much better reflection of its performance in this field. Yildirim’s publications have attracted 421 citations, excluding the self-citations. Mathematics is not a very citation dense field, so this level of citations is able to strongly influence both the PP(top10%) and the MNCS indicators.
An investigation into Yildirim’s publications has not yet started, as far as we know. But suspicions of fraud and plagiarism are rising, both in Turkey and abroad. One of his publications, in the journal Mathematical Physics, has recently been retracted by the journal because of evident plagiarism (pieces of an article by a Chinese author were copied and presented as original). Interestingly, the author has not agreed with this retraction. A fair number of Yildirim’s publications have been published in journals with a less than excellent track record in quality control. The Elsevier journal Computer & Mathematics with Applications (11 articles by Yildirim) has recently retracted an article by a different author because it turned out to have “no scientific content”. Actuallly, it was an almost empty publication. According to Retraction Watch, the journal’s editor Ervin Rodin has been replaced at the end of last year. He was also relieved from his editorial position at the journal Applied Mathematics Letters – An International Journal of Rapid Publication, another Elsevier imprint. Rodin was also editor of Mathematical and Computer Modelling, in which Yildirim published 5 articles. The latter journal currently does not accept any submissions “due to an editorial reconstruction”.
How did Yildirim’s publications attract so many citations? His 65 publications are cited by 285 publications, giving in total 421 citations. This group of publications has a strong internal citation traffic. They have attracted almost 1200 citations, of which a bit more than half is generated within this group. In other words: this set of publications seems to represent a closely knit group of authors, but they are not completely isolated from other authors. If we look at the universities citing Ege University, none of them have a high rank in the Leiden Ranking with the exception of Penn State University (which ranks at 112) that has cited Yildirim once. If we zoom in on mathematics and computer science, virtually all of the citing universities do not rank highly either, with the exception of Penn State (1 publication) and Gazi University (also 1 publication). The rank position of the last university, by the way, is not so reliable either, as indicated by the stability interval that is almost as wide as in the case of Ege University.
The bibliometric evidence allows for two different conclusions. One is that Yildirim is a member of a community which works closely together on an important mathematical problem. The alternative interpretation is that this group is a distributed citation cartel which not only exchanges citations but also produces very similar publications in journals that are functioning mainly as citation generating devices. A cursory look at a sample of the publications and the way the problems are formulated seems to support the second interpretation more than the first.
But from this point, the experts in mathematics should take over. Bibliometrics is currently not able to properly distinguish sense from nonsense in scientific publications. Expertise in the field is required for this task. We have informed the rector of Ege University that the ranking of his university is doubtful and requested more information from him about the position of the author. We have not yet received a reply. If Ege University wishes to be taken seriously, it should start a thorough investigation of the publications by Yildirim and his co-authors.
If you see other strange rankings in our Leiden Ranking or in any other ranking, please do notify us. It may help us create better tools to uncover fraudulent behaviour in academic scholarship.
The Citation Culture 
July 4, 2013 at 12:30 PM
Thanks for this detailed post. It seems that Ahmet Yıldırım’s PhD was recently revoked since it was a direct translation of a book of Ji-Huan He who is also quite a questionable figure in academia (http://elnaschiewatch.blogspot.com.es/2011/02/ji-huan-he-loses-ijnsns.html). It also seems that he was dismissed from the university (again without any official statement).
Here is Ahmet Yıldırım’s PhD ‘thesis’: https://docs.google.com/file/d/0BxUoSj9K4YfeNDIwUUZGRWU1R2c/edit?pli=1
And this is Ji-Huan He’s book: https://docs.google.com/file/d/0BxUoSj9K4YfeZmZvdGpDQUVWY0E/edit?pli=1
July 4, 2013 at 1:31 PM
This is a very instructive blog. It teaches us about the benefits and limits of bibliometrics. It also teaches us about the possible perverse effects of ranking by citation scores. Who wouldn’t want to have a list of ‘journals that are functioning mainly as citation generating devices’?
July 4, 2013 at 5:11 PM
As a citizen of Turkey who is working in the US as a scientist, it breaks my heart to admit that Turkey is still in the dark ages of science with respect to the academical integrity and ethics.
The current situation of published thesis in Turkey gives a glimpse to the scientific deterioration and ethical tolerance:
http://subjektif.org/landscapes-from-turkish-academy/
Complete copy-paste of academical content is deemed “OK” in Turkey, and consequences of stealing from other scientists is next to nothing. That’s why it is not surprising that Ege University did not take action against Ahmet Yıldırım for more than five years, despite the warnings from within the department itself, moreover they could be so proud of being second in this ranking.
Thank you very much for writing this article.
July 4, 2013 at 6:47 PM
There is a strong temptation for Universities to hide research fraud in order to protect their international rankings.
For the last nine years I have been fighting to expose a case of financial and research fraud at Manchester University, UK.
But the truth is so potentially damaging, with the possibility that lives have been lost as result of fraudulent research, that nobody in the British research establishment wants to know.
Details of the fraud are published online at
http://www.abettermousetrap.co.uk/whistleblower-inventor-alleges-fraud-by-manchester-university
and
http://www.cheshire-innovation.com/sali/pedsali.htm
July 13, 2013 at 10:26 PM
This unfotunate event was noticed with inquiries by Ege University in 2010. So that in 2011, ege university cut relationship with Ahmet Yıldırım. He appealed to the court for this decide however universitiy was justified.
In 2012, Mr.Yıldırım’s PhD degree was also canceled.
Ege University has made more than its share in this event. And i think, this university is very uneasy about it bacause Ege University commonly ranks close to the top among research universities in Turkey.
August 28, 2013 at 8:34 PM
[…] Paul Wouters writes: […]
September 4, 2013 at 9:33 AM
Is there any documentary evidence available that his PhD was revoked? or of his leaving Ege University?
September 23, 2013 at 7:53 AM
This kind of discussion serves some useful purpose in raising good questions of ethics in scientific research. For a long time, I too have worried about the risks associated with using bibliometric indices in quality assessments. In some institutions, monetary rewards have been attached to these, and that is a recipe for disaster. However, one must avoid generating a witch-hunt in which named persons can easily be assumed to be guilty. I cannot say that any of the statements here are factually incorrect. However, they should not be assembled in such a way as to cast aspersions. As an editor myself, I can say that the incidence of attempted plagiarism and fraud has increased dramatically. I can honestly say that Elsevier staff with whom I have been associated have worked hard to detect it and stamp it out, no matter who are the authors. The increased volume of submissions in recent years has generated an enormous workload, increasing the chances of some pathogens filtering through. Ervin Rodin in particular has borne a heavy load. He has given good honest service to the applied math community over many years. His departure has left a large backlog with long delays, including plagiarism checks. These long delays and associated effects on reputation are partly the reason why Elsevier has decided to close the journal.
Philip Broadbridge
President ANZIAM
Acting Editor-in-Chief
Mathematical and Computer Modelling