From: mk_thisisit
The Polish system for assessing scientists has been observed to exhibit pathologies that lead to the acceptance of low-quality or even nonsensical articles by journals otherwise considered high-ranking 00:00:00. This phenomenon is a direct consequence of the point-based evaluation of scientific work 00:00:27.
The “Trolling” Experiment
A Polish professor conducted an “experiment” to protest and highlight the flaws in the national system of assessing scientific achievements 00:00:40, specifically the evaluation of scientists’ work 00:01:17. He submitted a series of “senseless articles” to several scientific journals that were highly ranked on the Ministry of Science’s score list, including prestigious international publications like Science and Nature 00:01:23.
Content of the Articles
Despite their high standing, several of these nonsensical articles were accepted for printing, seemingly after serious reviews 00:01:48. Examples of these articles included:
- A piece on the hobby of young Karol Wojtyła (later Pope John Paul II) about collecting matches 00:02:01.
- An article about the influence of Catholic radio broadcasts on the moral attitudes of young people 00:02:09.
The professor, a chemist by profession, co-authored these articles with a musicologist who also held a doctorate in theoretical physics and a master’s degree in the economics of socialism 00:04:11. Editors reportedly verified their qualifications and considered them suitable authors for the topics 00:03:20.
Points and Effort
Although the professor did not report these articles to the system, internet users calculated that they could have yielded approximately 660 points 00:02:30. In contrast to genuine scientific research, which often requires years of serious theoretical and experimental work 00:02:44, the fictitious articles took only “d weeks of work” 00:03:04.
Public Reaction
A “bomb went off” in Poland following these revelations 00:03:37. The situation gained widespread attention in Polish media due to its comical and exotic nature: serious, highly-scored journals were publishing what amounted to a “joke” 00:04:33. The articles were not even linguistically correct 00:04:56.
Root Causes of the Problem
The professor’s actions were a protest against the point-based system of assessing scientific achievements 00:05:01. In this system:
- Work is assessed by points collected for each published article 00:05:16.
- Each journal title is assigned a specific number of points 00:05:25.
- More points generally lead to greater success 00:05:27.
However, points were assigned arbitrarily and almost randomly 00:05:32. This led to absurd situations where:
- Globally recognized journals like Science or Nature had the same number of points as Polish mid-range or even questionable publishers 00:08:39.
- Niche journals, practically unnoticeable on the international scientific market and often publishing only in Polish, received equal points to top-tier international journals 00:09:13.
This system encouraged scientists, especially younger ones seeking scholarships and career development, to prioritize points over genuine scientific merit 00:06:03. Discussions in laboratories shifted from scientific discourse to strategizing which journal to send articles to for maximum points 00:06:19.
The professor noted that some journals that topped the point list, such as Toruń Studies, Catholic Pedagogy, or Chemistry Metrology Education, were not previously considered significant on the scientific market 00:07:00.
Impact on Polish Science
The effect of such a system on Polish science is overwhelmingly negative 00:09:24:
- Frustration and Disappointment: It causes negative feelings among scientists 00:09:37.
- Discourages International Collaboration: Equating top international journals with low-quality local ones is unfair and hinders engagement with the global scientific community 00:10:36. Science is inherently international, fostering exchange of ideas and data 00:10:15.
- Decline in Quality: Many scientists, seeing that a small, weak, niche journal counts equally, choose the easier path instead of striving for publication in the best global journals 00:11:00. This leads to a sharp decline in the quality of science practiced in Poland 00:11:00.
- Quantity Over Quality: It incentivizes publishing numerous papers regardless of content. Cases exist of Polish scientists publishing 100-200 papers in 2-3 years, a physically impossible feat if texts were properly written and proofread, especially alongside teaching and administrative duties 00:11:23.
- Rise of “Paper Mills”: The system fosters “underground companies” or “paper mills” where groups of scientists send similar, often self-citing articles en masse to little-known publishing houses that publish anything for a fee, often without editorial control or review 00:12:45. These dubious journals often held high point values in Poland 00:13:06.
Broader Consequences
The system also contributes to the export of Polish scientific talent 00:29:41. Polish scientists who leave the country often create breakthrough achievements abroad due to a lack of financial resources or frustration with the hierarchical system at home 00:30:04. While this is not a loss for global science, it is a significant loss for Poland’s economy and education system, as the best minds leave 00:20:05.
Signs of Change and Future Directions
The professor’s protest did lead to some initial changes, with one of the previously lenient journals, Toruń Studies, rejecting a subsequent haphazard article due to bibliographical errors 00:13:49.
The Minister of Science acknowledges the widespread issue of the point system within the academic community 00:26:08. While current evaluations are ongoing, there is a clear intention to rethink the philosophy of evaluation for Polish science and universities 00:26:32. This will likely be addressed during amendments to the Act on Higher Education 00:26:40.
Key areas for potential change include:
- Shift from Patent Points to Implementation: Instead of awarding points for patents (many of which are “reinventing the wheel” just for points 00:27:10), the focus should be on the actual implementation of patents 00:27:00.
- Focus on Innovation and Promotion: The Ministry has established a Department of Innovation to promote achievements and innovative implementations, aiming to be more helpful to the scientific community 00:28:00.
- Investment in Risky, Breakthrough Solutions: The National Centre for Research and Development should be reformed to invest in innovative and higher-risk solutions, encouraging groundbreaking projects that officials might currently dismiss 00:31:07.
- Prioritizing Resources: Current priorities include national security, cybersecurity, and armaments, which will direct funds to related scientific research 00:31:37.
- Inter-Institutional Cooperation: Encouraging universities and institutes to cooperate and share expensive equipment, as much of the existing infrastructure is underutilized 00:32:44. A “census of nature” might be conducted to inventory national scientific resources 00:33:32.
The general sentiment is that the future evaluation system should be non-point-based, or if points remain, they should be based on an independent, international bibliometric system that classifies journals and assesses their rank 00:16:07. However, even relying solely on journal prestige is flawed, as good journals can still publish erroneous work 00:16:40. The challenge remains in finding a fair method to evaluate diverse fields of science, as comparing disciplines like mathematics, physics, and medieval Portuguese literature is illogical 00:15:21.
Despite these challenges, the conversation and increased awareness signify a will and desire within the scientific community and the Ministry to seriously discuss and change the problematic system 00:27:28.