counting scientific publications as metric for scientific quality is dumb

Scientific institutions¹ currently base a large part of their internal evaluation, their comparison to others, and their hiring decisions on counting publication (with a number of different scorings).

And this is dumb.

On the surface this causes pressure to publish as many papers as possible² which drives down quality of publications to the lowest standard reviewers accept.³ And it strengthens a hierarchy of publishers, where some publications are worth more than others based on the name of the journal. That simplifies funding decisions. But makes them worse. And it creates an incentive to get a maximum of prestige with a minimum of substance.

But publications are how scientists communicate. Adding another purpose to them reduces their value for communication — and as such harms the scientific process. Add to this that the number of scientists is rising and that scientific communication is already facing a scalability crisis, and it’s clear that counting publications as a metric of value is dumb. That’s clear to every scientist I ever talked to in person (there are people in online-discussions who disagree).

That it is still done shows that this pressure to publish is a symptom of an underlying problem.

This deeper problem is that there is no possible way to judge scientific quality independently. But universities and funders want competition (by ideology). Therefore they crave metrics. But »When a measure becomes a target, it ceases to be a good measure.«

Science is a field where typically only up to 100 people can judge the actual quality of your work, and most of these 100 people know each other. Competition does not work as quality control in such a setting; instead, competition creates incentives for corruption and group-thinking. Therefore the only real defense against corruption are the idealism of scientific integrity (“we’re furthering human knowledge”) and harsh penalties when you get caught (you can never work as scientist again).

But if you have to corrupt your communication to be able to work in the first place, this creates perverse incentives to scientists⁴ and will on the long run destroy the reliability of science.

Therefore counting publications has to stop.

Science is a field where constant competition undermines the core features society requires to derive value from it. Post-doc scientists proved over more than a decade that they want to do good work. Their personal integrity is what keeps science honest. Scientific integrity is still prevailing in most fields against the corrupting incentives from constant forced competition, but it won’t last forever.

If we as society want scientists we can trust, if we want scientific integrity, we have to move away from competition and towards giving more people permanent positions in public institutions; especially for science staff, the people who do the concrete work; the people who conduct experiments, search for mathematical proofs, and model theories.

Scientific integrity, personal motivation to do good work for the scientific sub-community (of around 100 people), and idealism (which can mean to contradict the community), along with the threat of being permanently expelled for fraud, are the drivers which produce good, reliable scientific results.

To get good, reliable results in science, the most important task is therefore to ensure that scientists do not have to worry too much about other things than their scientific integrity, their scientific community, and their idealism. Because it is only the intrinsic motivation of scientists which can ensure the quality of their work.