Experimenter

A package for easily running experiments for different parameters and saving the results in a centralised database

Package Features

• Create a local SQLite database to store the results of your experiment, removing the need to keep track of 1000s of results files for each parameter configuration.
• Provides a standard structure for executing code across a range of parameters.
• Provides saving of results into the database using standard Julia types.
• Promotes writing a script that can be easily committed to a Git repository to keep track of results and parameters used throughout development.
• Provides an @execute macro that will execute an experiment (consisting of many trials with different parameters). Can execute serially, or in parallel with a choice of multithreading or multiprocessing or even MPI mode.
• Provides an easy way to execute trials across a High Performance Cluster (HPC).
• Automatically skips completed trials, and provides a Snapshots API to allow for partial progress to be saved and reloaded.

Head over to Getting Started to get an overview of this package.

Manual Outline

• Getting Started
• • Opening the database
  • Defining the work we want to process
  • Creating an experiment
  • Examining the trials of an experiment
  • Executing an experiment
  • Getting the results
  • Re-running failed trials
  • Saving part way
  • What is an Experiment?
• Running your Experiments
• • Single Node Parallel
  • Heterogeneous Execution
  • MPI Execution
• Distributed Execution
• Cluster Execution
• • MPI
  • Experiment file
  • SLURM
• Store
• • Usage
• Custom Snapshots

Check out the API at Public API.