About reproducibility

Reproducibility of research results is a fundamental property of high-quality research. (Royal Netherlands Academy of Arts and Sciences)

However, recent failures to replicate scientific work highlight the necessity for new initiatives to ensure reproducibility. CheckResearch.org is a community effort to validate and preserve scientific research by offering a platform to increase the reproducibility of research findings. It focuses on the domain of computer science, in which software and data are typically the main artifacts.

Both, the authors and the readers of scientific research benefit from an independent validation. The community checks if their approaches are described well enough for reproduction. Furthermore, their provided artifacts are tested for completeness and executability. In addition, varying setups and new data sources may deliver new insights to the authors’ approaches. Finally yet importantly, their approaches are presented to the public. Authors can also use CheckResearch.org to demonstrate step-by-step how they conducted their initial research, as template for the public. The public profits by independent validations of scientific research and particularly by a tested explanation how the scientific results can be reproduced thanks to the effort of individuals from the community. It hence fosters understanding and adoption of the scientific work. Moreover, we invite reviewers to share their evaluation results (of published work).

CheckResearch.org focuses on increasing the reproducibility of scientific results; it is not a mere artifact repository. Hence, the original artifacts can be gathered from existing repositories (such as findresearch.org), publishers or by contacting the authors directly.

The terms reproducibility and replicability are often used synonymous and the exact meaning of these terms is often subject to discussion. We define reproducibility and replicability according to the definition of Pellizzari et al.: