Malte Elson, a psychologist at the University of Bern, is on a mission to revolutionize the way scientific errors and inaccuracies are detected and corrected. He believes that the current system is flawed and ineffective, citing a well-known case where errors in a highly influential paper by economists Carmen Reinhart and Kenneth Rogoff went unnoticed for years until a diligent PhD student uncovered them.
The Impact of Errors in Scientific Research
Elson emphasizes the importance of identifying errors in scientific research, especially when these errors have far-reaching consequences. In the case of the Reinhart and Rogoff paper, which promoted austerity measures in Europe, the errors led to misguided policy decisions that could have been avoided if proper error detection mechanisms were in place.
The ERROR Project: A New Approach to Error Detection
To address this issue, Elson and his colleagues launched the Estimating the Reliability and Robustness of Research (ERROR) project. The project aims to pay researchers to review highly cited psychology papers for errors in code, statistical analyses, and reference citations. By incentivizing error detection, the ERROR project hopes to create a more systematic and effective way to identify and correct scientific inaccuracies.
Subheading: The ERROR Project in Action
Since its launch, the ERROR project has made significant progress in identifying errors in published papers. Reviewers are paid for each paper they check, with bonuses for finding errors that result in major corrections or retractions. This incentive-based approach is modeled after ‘bug bounty’ programs in the tech industry, where hackers are rewarded for finding vulnerabilities in software.
The Impact of the ERROR Project
The ERROR project has already had a positive impact on the psychology research community. By uncovering errors in highly cited papers, the project has helped to improve the reliability and robustness of research in the field. Authors are also compensated for their cooperation in the review process, creating a collaborative environment that benefits both researchers and the scientific community as a whole.
Challenges and Future Expansion
Despite its success, the ERROR project faces challenges in gaining author cooperation and finding reviewers with the necessary technical expertise. However, Elson and his team are actively working to expand the project to other disciplines, such as artificial intelligence and medical research. By demonstrating a scalable and transferable model for error detection, the ERROR project hopes to inspire other researchers to implement similar practices in their own fields.
Subheading: The Role of Funding in Error Detection
Elson believes that funders of research should also play a role in supporting error detection efforts. By funding error reviews of the research they support, funders can help to improve the quality and reliability of scientific research. Elson has already engaged with research foundations and private funders to discuss the importance of investing in error detection mechanisms.
In conclusion, the ERROR project represents a groundbreaking approach to error detection in scientific research. By incentivizing reviewers to identify errors in published papers, the project aims to improve the reliability and robustness of research in various disciplines. With continued support and expansion, the ERROR project has the potential to revolutionize the way scientific errors are detected and corrected, ultimately enhancing the quality of research worldwide.