replicability

Specifying and executing multiverse analysis

A Python library to determine what exactly the user ran at the command line, along with default values, and any transformations that happened after parsing arguments.

Implementation of metaheuristic optimization methods in Python for scientific, industrial, and educational scenarios. Experiments can be executed in parallel or in a distributed fashion. Experimental results can be evaluated in various ways, including diagrams, tables, and export to Excel.

A Python package enabling the development of robust automation scripts that are subdivided into stages.

Keep track of your results

Resources on the Artifact Evaluation (AE) Process

zcurve R package for assessing the reliability and trustworthiness of published literature with the z-curve method

TriScale software

A curated list of publicly available datasets for replication studies.

A Python Interface to Reproducibility Measures of System-Oriented IR Experiments

A flexible evaluation platform to enable researchers to conduct replicable subjective evaluation

A framework-agnostic deep learning package and experiment manager

Special Issue: The Potential of Notebooks for Scientific Publication: Reproducibility, and Dissemination

Position Paper for the J-STARS Special Issue on Reproducible Research in Remote Sensing

Code repository accompanying "Assessing data availability and research reproducibility in hydrology and water resources"

Repository for R package and website of replication diagnosis tools

Git copy of our Overleaf project for the preprint of Nine Best Practices for Research Software Registries and Repositories: A Concise Guide

[ECIR 2022] Automation of Citation Screening for Systematic Literature Reviews Using Neural Networks: A Replicability Study

Wide Residual Networks in Keras and PyTorch

A Python package that allows you to interact with the shell, while logging various metadata, statistics, and trace information.