How evolution can change science for the better

Science is society’s best method for understanding the world. Yet many scientists are unhappy with the way it works, and there are growing concerns that there is something “broken” in current scientific practice. Many of the rules and procedures that are meant to promote innovative research are little more than historical precedents with little reason to suppose they encourage efficient or reliable discoveries. Worse, they can have perverse side-effects that harm both science and scientists. A well-known example is the general preference for positive over negative results, which creates a “publication bias” that gives the false impression that certain effects exist where in reality the dissenting evidence simply fails to be released.

Arizona State University researchers Thomas Morgan and Minhua Yan, working with ASU graduate Leonid Tiokhin, now at University of Technology Eindhoven in the Netherlands, have developed a new model, published this week in Nature Human Behavior, to better understand the challenges facing the scientific process and how we can make it better. They focused on the “priority rule”: the tendency for the first scientist to document a finding to be disproportionately rewarded with prestige, prizes and career opportunities while those in second place get little to no recognition.

Winner takes all

Many scientists have sleepless nights worrying about being “scooped” — fearing that their work won’t be considered “novel” enough for the highest-impact scientific journals because a different group working on the same topic manages to publish first. The priority rule has been around for centuries. In the 17th century, Newton and Leibniz haggled over who invented calculus. And in the 19th century, Charles Darwin rushed to publish of “The Origin of Species” to avoid being scooped by Alfred Russel Wallace.

“Rewarding priority is understandable and has some benefits. However, it comes at a cost,” says Tiokhin. “Rewards for priority may tempt scientists to sacrifice the quality of their research and cut corners.”

“The idea is that competition encourages scientists to work hard and efficiently, such that discoveries are made quickly. But if everyone is working hard, and you need to come in first to be successful, then there’s a temptation to cut corners to maximize your chances, even if it means the science suffers,” said Morgan, a research affiliate with the Institute of Human Origins and associate professor with the School of Human Evolution and Social Change at Arizona State University.

This is partly why some academic publishers, such as PLOS and eLife, now offer “scoop protection,” allowing researchers to publish findings identical to those already published within a certain timeframe. The problem is that science and publishers currently don’t have a good idea about whether these reforms make sense.

Modelling the priority rule

To figure out how exactly the preference for priority affects science, and whether recent reforms offer any solution for its potential drawbacks, the collaborators developed an “evolutionary agent-based model.” This computer model simulates how a group of scientists investigate or abandon research questions, depending on their own results and the behavior of other scientists they compete against.

“The benefit of an evolutionary simulation is that we don’t need to specify in advance how scientists behave. We just create a world in which success is rewarded, and we let selection figure out what kinds of behavior this favors,” said Morgan. “We can then vary what it means to successful — for instance, whether or not it’s critical to come first — and see how selection changes the behavior of scientists in response. We can also measure the benefit to society — are scientists being efficient? Are their findings accurate? And so on.”

No panacea

The researchers found that a culture of excessive rewards for priority can have harmful effects. Among other things, it motivates scientists to conduct “quick and dirty” studies, so that they can be first to publish. This reduces the quality of their work and harms the reliability of science as a whole.

The model also suggests that scoop protection, as introduced by PLOS and eLife, works.

“It reduces the temptation to rush the research and gives researchers more time to collect additional data,” said Tiokhin. “However, scoop protection is no panacea.”

This is because scoop protection motivates some scientists to continue with a research line even after several results on that topic have been published, which reduces the total number of research questions the scientific community can address.

The ‘benefit’ of inefficiency

Scoop protection reforms in themselves, while helpful, are not sufficient to guarantee high-quality research or a reliable published literature. The model also shows that even with scoop protection, scientists will be tempted to run many small studies if new studies are cheap and easy to set up and the rewards for negative results are high. This suggests that measures that force scientists to invest more heavily in each study, such as asking scientists to preregister their studies or get their research plans criticized before they begin collecting data, can help.

“We also learned that inefficiency in science is not always a bad thing. On the contrary — inefficiencies force researchers to think twice before starting a new study,” said Tiokhin.

Another option is to make large-scale data collection so straightforward that there is less incentive to skimp on data, alternatively, reviewers and journals could be more vigilant in looking out for “underpowered” studies with small sample-sizes.

Metascience

This project is an example of metascience, the use of the scientific method to study science itself.

“It was a great pleasure to be part of this project. I got to use my modeling skills not only to make specific scientific discoveries, but also to shed light on how the scientific procedure itself should be designed to increase research quality and credibility. This benefits the whole scientific community and ultimately, the whole society,” said Yan, a graduate student in the School of Human Evolution and Social Change.

###

Article published in Nature Human Behaviour, “Competition for priority harms the reliability of science but reforms can help,” Leonid Tiokhin, Minhua Yan, Thomas J.H. Morgan.