From: mk_thisisit
The point-based system for evaluating scientific achievements in Poland is widely recognized as a significant problem, often leading to detrimental outcomes for research quality and academic integrity [00:00:00], [00:00:27], [00:00:48]. This system assigns specific point values to publications based on the journal title, with higher points theoretically signifying greater achievement [00:05:19]. However, this method is criticized for its arbitrary and nearly random assignment of points [00:05:32].
The “Trolling” Experiment
A Polish professor, Konrad, decided to deliberately challenge the system by submitting nonsensical articles to highly-rated (by the Polish point system) scientific journals, including some internationally recognized ones [00:01:19], [00:01:42]. His “protest against the system of assessing scientific achievements” highlighted major flaws [00:05:01].
The articles submitted included topics such as:
- The hobby of young Karol Wojtyła (later Pope John Paul II) involving collecting matches [00:02:01].
- The influence of Catholic radio broadcasts on the moral attitudes of young people [00:02:09].
Several of these nonsensical articles were accepted for printing, seemingly after serious reviews [00:01:46]. The authors’ qualifications (a chemist and a musicologist with a doctorate in theoretical physics and a master’s in the economics of socialism) were checked and deemed appropriate by the editors, despite the wildly unrelated subject matter [00:03:06], [00:04:11]. One article, poorly structured with undefined statistical studies and data analysis, was initially ridiculed on social media before the authors’ identities were discovered, turning the situation into a widespread media story across Poland due to its comical and exotic nature [00:03:52], [00:04:33].
The professor estimates that if reported, these fictitious articles could have earned around 660 points, which would equate to several years of serious theoretical and experimental scientific work for a chemist [00:02:37]. The time spent on these fictitious articles was only about “d weeks of work” [00:03:04].
Pathologies of the Polish Scientific Assessment System
The core issue lies in the point system itself, which is one of the major pathologies of the Polish system of assessing scientists [00:00:00], [00:05:51].
- Arbitrary Point Assignment: Points are often assigned arbitrarily, leading to situations where internationally recognized, high-quality journals have very few points, while journals of questionable quality have many [00:05:32], [00:05:36], [00:08:39]. Some niche Polish journals, sometimes publishing only in Polish, received the same number of points as prestigious international journals like Science or Nature [00:09:13], [00:09:18].
- Focus on Quantity over Quality: The system incentivizes scientists, especially younger ones seeking scholarships and career development, to accumulate as many points as possible [00:06:03], [00:06:09]. This shifts focus from scientific discussions to strategizing which journal to submit to for maximum points [00:06:19], [00:06:22].
- Rapid Publication Rates: The system enables some scientists to publish hundreds of papers within two to three years, a quantity that is physically impossible to produce with genuine, high-quality work, especially when also juggling teaching and administrative duties [00:00:08], [00:11:23], [00:11:55].
- “Paper Mills”: The existence of “paper mills” is noted, where groups of scientists collaborate to send very similar articles en masse to lesser-known publishing houses that are often highly scored in Poland. These publishers frequently accept articles for a fee, operating practically without editorial control or review [00:12:48], [00:13:06], [00:13:14], [00:13:35]. This highlights the acceptance of low-quality articles by high-ranking journals (by point score) in the system.
Impact of the point system on the quality of Polish science
The system is seen as having only negative effects, leading to frustration and disappointment among scientists [00:09:30]. It promotes publishing in easy-to-access, weak niche journals rather than striving for global top-tier publications, because both count equally towards achievements [00:11:00]. This may lead to a sharp decline in the overall quality of science practiced in Poland [00:11:00].
“equating the best journals where everyone would like to publish… with journals where you can publish any nonsense written on your knees is unfair” [00:10:36].
Comparisons and Consequences
International Perspective
In Western scientific communities, achievements are assessed based on successes like publications in the best conferences and top-tier journals such as Nature. There are no point-based lists [00:08:00].
The “National Science” Delusion
The concept of “national science” is likened to the Soviet Union’s promotion of “Soviet science” under Stalin [00:10:07]. True scientific progress thrives in an international environment with a free exchange of ideas, concepts, and data [00:10:17]. The current system, by valuing domestic publications equally with international ones, discourages global engagement.
Export of Polish Scientific Talent
The system contributes to the export of Polish scientific talent [00:18:18], [00:29:41]. Top scientists leave Poland because they lack sufficient funding for development or are frustrated by the hierarchical system [00:30:07]. While this is not a loss for global science, it is a significant loss for Poland’s economy and education system, as the best minds leave, and their innovations contribute to other countries’ economies [00:20:05].
Calls for Scientific and Academic Reform
There is recognition that the system needs to change [00:00:33], [00:25:58]. The Minister of Science acknowledges the widespread concern about the point system across academic centers [00:26:08].
Proposed Changes:
- New Evaluation Philosophy: Discussions are underway for a new point scale or an entirely non-point-based system [00:15:10], [00:16:07], possibly defined in an amendment to the Act on Higher Education [00:26:40].
- Commercialization and Patents: The focus should shift from merely accumulating points for patents to evaluating the actual implementation of patents [00:26:58].
- Innovation and Promotion: The Ministry of Science has established a Department of Innovation to promote innovation and scientific achievements, and to better support the scientific community [00:28:00], [00:28:12].
- Risk Investment: The National Centre for Research and Development (NCBR) should be reformed to invest in risky, innovative solutions, which is crucial for fostering breakthrough achievements [00:31:07].
- Resource Optimization: A “census of nature” or inventory of scientific equipment across Poland is suggested to ensure better utilization of existing expensive machinery, promoting cooperation between institutes and universities rather than redundant purchases [00:33:32], [00:33:41], [00:34:00].
- Priorities: Current funding priorities lean towards security and cybersecurity, which will also direct funds towards Polish science and institutes in these areas [00:31:37].
Broader Challenges in Polish Science
Funding and Remuneration
Scientific progress and funding in Poland are hampered by inadequate remuneration for scientists, especially PhD students, assistants, and adjuncts, forcing them to seek additional work or grants, diverting their focus from research [00:19:12], [00:19:31].
Public Engagement and Education
Public interest in science seems to be declining [00:21:10]. The education system, particularly at lower levels, is seen as capable of killing interest in science by focusing on rote learning instead of practical applications and accessible explanations of phenomena [00:22:31], [00:22:46]. More scientists need to engage in popularizing science, but the current point-based system consumes time that could be spent on such initiatives [00:24:25].
Recognition of Women in Science
There is a notable disparity in leadership roles, with project managers and university rectors predominantly being men [00:34:30], [00:34:39]. While no direct quotas will be introduced, the Minister believes that increased activity among women will naturally lead to more women in science and leadership positions [00:35:36]. He asserts that the quality of teaching and research should be the primary criterion, where women will excel [00:36:00].
In conclusion, the point-based evaluation system in Polish science presents a complex and deeply problematic issue, driving perverse incentives and hindering genuine scientific advancement and international collaboration. While changes are being discussed, the path forward remains challenging [00:15:01], [00:32:28].