Improving Code Quality Through Peer Review
Researchers in the field of earth and planetary science now have a new tool at their disposal to enhance the quality of their work. By using Code Ocean, authors can make the peer review process of their code smoother and more transparent.
In the world of scientific research, trust is built on transparency and reproducibility. However, when it comes to the intricate computer code that forms the backbone of many research papers, things can get murky. Not only do researchers face challenges in documenting and sharing their code effectively, but peer reviewers and readers may struggle to install, run, and test the code themselves. Addressing these issues is crucial in improving the reproducibility of computational research.
A decade ago, Nature Portfolio journals introduced a policy requiring authors to provide a statement on the availability of their computer code. This statement had to outline how and where the code could be accessed, along with any restrictions on access. Despite the benefits of code sharing for reproducibility, many researchers faced obstacles in making their code available. As a result, authors often stated that their code was not accessible. Nevertheless, the policy aimed to push geoscientists towards a more transparent future.
In 2018, the policy evolved to actively promote the peer review of code for papers where the code played a central role. Authors were now obliged to make their custom code available to editors and reviewers upon request. However, implementing code peer review posed challenges, and until recently, Nature Geoscience did not have a consistent system for conducting code peer review.
Since July, authors submitting primary research articles to Nature Portfolio journals have had the option to share their code for peer review on Code Ocean. This cloud-based platform allows authors to set up a “Compute Capsule” for their code, which reviewers can access confidentially. Code Ocean enables reviewers to run and evaluate the code anonymously, without the need to install it themselves. If a paper is published, the capsule becomes publicly available with a citable DOI. Even if a paper is not accepted, authors can choose to maintain the capsule. The service is free for both authors and reviewers.
While the use of Code Ocean is encouraged, authors also have the flexibility to share their code through other repositories or as supplementary material. Regardless of the method chosen, authors must provide information on how their code can be accessed during the submission process. It is not currently mandatory to make custom code publicly available, but authors are strongly encouraged to do so. The trend of citing publicly available code in research papers is on the rise, reflecting a shift towards greater transparency in the research community.
Nature Portfolio aims to adapt its policies to align with the evolving expectations of researchers. The collaboration with Code Ocean to facilitate code sharing and peer review signifies a step in that direction. By embracing tools like Code Ocean, researchers can enhance the quality and reproducibility of their work, ultimately contributing to the advancement of scientific knowledge in the field of earth and planetary science.