Why reproducibility?

By reproducibility, we refer to the main principle of the scientific method: research findings must be replicable through independent experimentation or data analysis. The results should be consistent with those of the original study, within acceptable error margin.

In modern scientific research, software plays a central role in data processing, analysis, and experimentation, introducing additional challenges to reproducibility. It is no longer sufficient to merely describe the methods used in an experiment. Access to the source code is essential - but not sufficient on its own.

Geoscientific Model Development (GMD) Guidelines

The GMD journal explicitly states the importance of reproducibility in its editorial guidelines. In {cite:t}gmd_2019, it introduced editorial guidelines regarding code and data policies. For example:

(...) it is not sufficient that the source code is provided. It is also necessary to have access to all the input data (...) and all model configuration files are provided.

Additionally:

(...) challenge (...) occurs where model inputs or outputs have been manually processed by an author. (...) nobody, not even the author, can definitively know (...) how the results came about.

All figures and tables must be scientifically reproducible from the scripts.

Why is reproducibility essential? Because it is a core principle of the scientific method and a requirement enforced by scientific journals.

Journals enforce reproducibility against archived releases (which is great!)

---

Why notebooks?

In this paper, we will present utility package and practical recommendations to support reproducible scientific research within the context of research software engineering (RSE). These tools are intended for use with scientific notebooks contained in code repositories. In this section, we explore the growing use of notebooks in research, based on statements from articles published in Nature.

Why Jupyter is data scientists’ computational notebook of choice

<img src="img/nature.svg.webp" width=500 class="center">

Nature 563 (toolbox): {cite:t}perkel_2018

We read:

We went from Jupyter notebooks not existing some six years ago to in essence everybody using them today.

However, the same paper also highlights challenges:

(...) difficult to organize code logically, break it into reusable modules and develop tests to ensure the code is working properly

---

Reactive, reproducible, collaborative: computational notebooks evolve

<img src=img/Nature2021.webp width=500>

Nature 593: {cite:t}perkel_2021

Three years later, they published:

A 2019 study found that just 24% of 863,878 publicly available Jupyter notebooks on GitHub could be successfully re-executed, and only 4% produced the same results.¹

Due to notebooks wide presence in scientific research, it is important to address the challenges they present.

---

Can we do even better?

Journals require that results are reproducible using the code and data working at the time of publication. This necessitates using specific package versions to ensure consistent behavior. Hence, we propose going further.

The reproducibility maintained with ongoing developments

<img src="img/signs-post-solid.svg" width=50 alt="next"> In next sections you can find solutions developed in our packages.

Footnotes

1. {cite:t}pimentel_2019 ↩