Command line¶
The dispatchatlas command is a thin layer over the library: every command
calls the public API and prints the result. It is also available as
python -m dispatchatlas.
Conventions¶
- Results go to standard output, as aligned text or, with
--json, as JSON. Log messages and errors go to standard error;--quietkeeps only warnings and errors, and--debugadds debug messages and the traceback of an error. -
A problem argument is a problem document (
.json), an instance file in a published format (see--format), or-for a problem document on standard input, so commands can be chained: -
--set KEY=VALUEsets a solver or family parameter. The value is read as JSON when it parses, else as text:--set tenure=10,--set local_search=false,--set tiers=edge,cloud. - The exit code is 0 on success, 1 when the operation fails (an invalid problem, a schedule that breaks a constraint, failed runs, a run that does not reproduce), 2 on a usage error, and 130 when the command is interrupted.
Commands¶
The help below is generated from the installed command line when this site is built.
usage: dispatchatlas [-h] [--debug] [-q] [--version] COMMAND ...
Reproducible benchmarking of scheduling algorithms.
options:
-h, --help show this help message and exit
--version show program's version number and exit
common options:
--debug show tracebacks of errors and debug messages
-q, --quiet log only warnings and errors
commands:
COMMAND
version Show the versions of DispatchAtlas, Python, and the optional backends.
solvers List the registered solvers and show their metadata.
benchmarks List the benchmark families and generate problems.
suites List the published benchmark suites and load their instances.
catalogs List the named instance catalogs and write them to disk.
solve Solve a problem with one solver and check the schedule it returns.
check Validate a problem, and a schedule against it.
campaign Validate, estimate, run, check, and replay experiment campaigns.
processes Launch, watch, list, and end processes; check and repair rows files.
analyze Write the analysis bundle of a campaign: tables, figures, and a
manifest.
references Export the bibliography of the implemented methods, as text or BibTeX.
Run 'dispatchatlas COMMAND --help' for the options of a command.
dispatchatlas version¶
usage: dispatchatlas version [-h] [--json] [--debug] [-q]
Show the versions of DispatchAtlas, Python, and the dependencies; which optional
backends are installed, with the extra that provides each; and the installed solver
and benchmark plugins.
options:
-h, --help show this help message and exit
common options:
--json print the result as JSON instead of text
--debug show tracebacks of errors and debug messages
-q, --quiet log only warnings and errors
dispatchatlas solvers¶
usage: dispatchatlas solvers [-h] COMMAND ...
List the registered solvers, built in and from plugins, and show what each one does,
its parameters, the problem features it models, and its sources.
options:
-h, --help show this help message and exit
commands:
COMMAND
list List the registered solvers.
show Show a solver's metadata, parameters, supported features, and citations.
dispatchatlas solvers list¶
usage: dispatchatlas solvers list [-h] [--json] [--debug] [-q] [--family F]
List the registered solvers.
options:
-h, --help show this help message and exit
--family F list only the solvers of this family
common options:
--json print the result as JSON instead of text
--debug show tracebacks of errors and debug messages
-q, --quiet log only warnings and errors
example:
dispatchatlas solvers list --family local-search
dispatchatlas solvers show¶
usage: dispatchatlas solvers show [-h] [--json] [--debug] [-q] ID
Show a solver's metadata, parameters, supported features, and citations.
positional arguments:
ID solver id
options:
-h, --help show this help message and exit
common options:
--json print the result as JSON instead of text
--debug show tracebacks of errors and debug messages
-q, --quiet log only warnings and errors
example:
dispatchatlas solvers show simulated-annealing
dispatchatlas benchmarks¶
usage: dispatchatlas benchmarks [-h] COMMAND ...
List the registered benchmark families, built in and from plugins, show their
parameters, and generate problems. Equal parameters and seed give the same problem.
options:
-h, --help show this help message and exit
commands:
COMMAND
list List the benchmark families.
show Show a family's summary, parameters, objectives, and citations.
generate Generate one problem of a family as a problem document.
dispatchatlas benchmarks list¶
usage: dispatchatlas benchmarks list [-h] [--json] [--debug] [-q]
List the benchmark families.
options:
-h, --help show this help message and exit
common options:
--json print the result as JSON instead of text
--debug show tracebacks of errors and debug messages
-q, --quiet log only warnings and errors
dispatchatlas benchmarks show¶
usage: dispatchatlas benchmarks show [-h] [--json] [--debug] [-q] ID
Show a family's summary, parameters, objectives, and citations.
positional arguments:
ID family id
options:
-h, --help show this help message and exit
common options:
--json print the result as JSON instead of text
--debug show tracebacks of errors and debug messages
-q, --quiet log only warnings and errors
example:
dispatchatlas benchmarks show job-shop
dispatchatlas benchmarks generate¶
usage: dispatchatlas benchmarks generate [-h] [--json] [--debug] [-q] [--seed N]
[--set KEY=VALUE] [--output FILE]
ID
Generate one problem of a family and print its problem document, or write it to a
file.
positional arguments:
ID family id
options:
-h, --help show this help message and exit
--seed N generator seed (default: 0)
--set KEY=VALUE set a family parameter; the value is read as JSON when it parses,
else as text
--output FILE write the problem to this file
common options:
--json print the result as JSON instead of text
--debug show tracebacks of errors and debug messages
-q, --quiet log only warnings and errors
example:
dispatchatlas benchmarks generate job-shop --seed 1 --set jobs=6 --set machines=4 --output js.json
dispatchatlas suites¶
usage: dispatchatlas suites [-h] COMMAND ...
List the published benchmark suites and load their instances. Generated suites
(Taillard's) are rebuilt from their published seeds; file suites are read from files
you download from the suite's source.
options:
-h, --help show this help message and exit
commands:
COMMAND
list List the published suites.
show Show a suite's instances, source, files to download, and citations.
load Load one instance of a suite as a problem document.
dispatchatlas suites list¶
usage: dispatchatlas suites list [-h] [--json] [--debug] [-q]
List the published suites.
options:
-h, --help show this help message and exit
common options:
--json print the result as JSON instead of text
--debug show tracebacks of errors and debug messages
-q, --quiet log only warnings and errors
dispatchatlas suites show¶
usage: dispatchatlas suites show [-h] [--json] [--debug] [-q] ID
Show a suite's instances, source, files to download, and citations.
positional arguments:
ID suite id
options:
-h, --help show this help message and exit
common options:
--json print the result as JSON instead of text
--debug show tracebacks of errors and debug messages
-q, --quiet log only warnings and errors
example:
dispatchatlas suites show taillard-flow-shop
dispatchatlas suites load¶
usage: dispatchatlas suites load [-h] [--json] [--debug] [-q] [--directory D]
[--output FILE]
ID INSTANCE
Load one instance of a suite and print its problem document, or write it to a file.
positional arguments:
ID suite id
INSTANCE instance name, in any case
options:
-h, --help show this help message and exit
--directory D directory holding the downloaded files of a file suite, searched
recursively
--output FILE write the problem to this file
common options:
--json print the result as JSON instead of text
--debug show tracebacks of errors and debug messages
-q, --quiet log only warnings and errors
examples:
dispatchatlas suites load taillard-flow-shop ta001 --output ta001.json
dispatchatlas suites load orlib-job-shop ft06 --directory downloads/orlib
dispatchatlas catalogs¶
usage: dispatchatlas catalogs [-h] COMMAND ...
List the built-in instance catalogs, show their entries, and write every instance of
one to a directory, with a manifest that records each file's SHA-256 digest, family,
parameters, and seed.
options:
-h, --help show this help message and exit
commands:
COMMAND
list List the built-in catalogs.
show Show the families and parameters of a catalog.
build Generate every instance of a catalog and write it with a manifest.
dispatchatlas catalogs list¶
usage: dispatchatlas catalogs list [-h] [--json] [--debug] [-q]
List the built-in catalogs.
options:
-h, --help show this help message and exit
common options:
--json print the result as JSON instead of text
--debug show tracebacks of errors and debug messages
-q, --quiet log only warnings and errors
dispatchatlas catalogs show¶
usage: dispatchatlas catalogs show [-h] [--json] [--debug] [-q] NAME
Show the families and parameters of a catalog.
positional arguments:
NAME catalog name
options:
-h, --help show this help message and exit
common options:
--json print the result as JSON instead of text
--debug show tracebacks of errors and debug messages
-q, --quiet log only warnings and errors
dispatchatlas catalogs build¶
usage: dispatchatlas catalogs build [-h] [--json] [--debug] [-q] --output DIR
[--seed N] [--overwrite]
NAME
Generate every instance of a catalog and write it to a directory as problems/<id>.json
with a manifest.json. Instance seeds derive from one root seed, so equal seeds give
identical files.
positional arguments:
NAME catalog name
options:
-h, --help show this help message and exit
--output DIR directory to write
--seed N root seed (default: 0)
--overwrite replace the problems of a directory that is not empty
common options:
--json print the result as JSON instead of text
--debug show tracebacks of errors and debug messages
-q, --quiet log only warnings and errors
example:
dispatchatlas catalogs build smoke --output instances/smoke --seed 1
dispatchatlas solve¶
usage: dispatchatlas solve [-h] [--json] [--debug] [-q]
[--format {fjs,orlib-flow-shop,orlib-job-shop,psplib}]
[--instance NAME] --solver ID [--set KEY=VALUE]
[--time-limit S] [--max-evaluations N] [--max-iterations N]
[--target V] [--patience N] [--seed N] [--output FILE]
PROBLEM
Solve a problem with one registered solver, check the schedule it returns against
every constraint, and print a summary. The full result, with the schedule and the
trace of improvements, is written with --output.
positional arguments:
PROBLEM problem document (.json), instance file in a published format,
or - for a problem document on standard input
options:
-h, --help show this help message and exit
--format {fjs,orlib-flow-shop,orlib-job-shop,psplib}
read PROBLEM in this instance format instead of the one its
extension implies
--instance NAME instance to read from a multi-instance OR-Library file
--solver ID solver id
--set KEY=VALUE set a solver parameter; the value is read as JSON when it
parses, else as text
--seed N seed of the run (default: 0)
--output FILE write the full result document here
common options:
--json print the result as JSON instead of text
--debug show tracebacks of errors and debug messages
-q, --quiet log only warnings and errors
stop criteria:
The first criterion met ends the run. Iterative solvers need a limit.
--time-limit S wall-clock seconds
--max-evaluations N schedules evaluated
--max-iterations N main-loop iterations
--target V stop once the objective reaches V
--patience N iterations without improvement
examples:
dispatchatlas solve js.json --solver heft
dispatchatlas solve js.json --solver tabu-search --set tenure=10 --max-evaluations 5000 --seed 3 --output result.json
dispatchatlas benchmarks generate flow-shop | dispatchatlas solve - --solver neh
dispatchatlas check¶
usage: dispatchatlas check [-h] [--json] [--debug] [-q]
[--format {fjs,orlib-flow-shop,orlib-job-shop,psplib}]
[--instance NAME]
PROBLEM [SCHEDULE]
Validate a problem: read it and check its structure. With SCHEDULE, also check the
schedule against every constraint of the problem and compute its objective values.
SCHEDULE is a schedule document, a result document written by 'dispatchatlas solve
--output', or a run record of a campaign. The exit code is 1 when the problem is
invalid or the schedule breaks a constraint.
positional arguments:
PROBLEM problem document (.json), instance file in a published format,
or - for a problem document on standard input
SCHEDULE schedule document, result document, or campaign run record to
check
options:
-h, --help show this help message and exit
--format {fjs,orlib-flow-shop,orlib-job-shop,psplib}
read PROBLEM in this instance format instead of the one its
extension implies
--instance NAME instance to read from a multi-instance OR-Library file
common options:
--json print the result as JSON instead of text
--debug show tracebacks of errors and debug messages
-q, --quiet log only warnings and errors
examples:
dispatchatlas check js.json
dispatchatlas check js.json result.json --json
dispatchatlas campaign¶
usage: dispatchatlas campaign [-h] COMMAND ...
Run experiment campaigns described by a TOML or JSON file: every solver on every
problem under every stopping protocol, with seeds shared across solvers, one record
per run, and resumption after an interruption.
options:
-h, --help show this help message and exit
commands:
COMMAND
validate Check a campaign file and plan its runs without running them.
estimate Count a campaign's runs and bound how long they can take.
run Run a campaign, resuming from the records it already has.
check Check a campaign directory before its results are used.
replay Run completed runs again and check that they reproduce.
dispatchatlas campaign validate¶
usage: dispatchatlas campaign validate [-h] [--json] [--debug] [-q]
[--check-compatibility] [--runs]
CONFIG
Read a campaign file strictly, load every problem, check every solver and parameter,
and plan the runs, verifying that the comparison is fair. Nothing runs.
positional arguments:
CONFIG campaign file (.toml or .json)
options:
-h, --help show this help message and exit
--check-compatibility
list every problem and solver pair that is excluded, with the
features the solver does not model, and every caveat, even
when no pair remains
--runs list every planned run with its id, problem, solver, protocol,
replicate, and seed, as 'campaign run' would write them to
plan.json; with --json, 'runs' is then that list in place of
its length
common options:
--json print the result as JSON instead of text
--debug show tracebacks of errors and debug messages
-q, --quiet log only warnings and errors
example:
dispatchatlas campaign validate campaign.toml --check-compatibility
dispatchatlas campaign estimate¶
usage: dispatchatlas campaign estimate [-h] [--json] [--debug] [-q] [--output DIR]
[--workers N] [--backend {sequential,process}]
CONFIG
Plan a campaign, count its runs, and bound their duration from the protocols' time
limits. With --output, runs already completed there are left out.
positional arguments:
CONFIG campaign file (.toml or .json)
options:
-h, --help show this help message and exit
--output DIR output directory of earlier runs of the campaign
--workers N worker processes, or 'auto' for the available processors minus
1 (default: the campaign's setting)
--backend {sequential,process}
run in worker processes, or one at a time in this process
(default: the campaign's setting)
common options:
--json print the result as JSON instead of text
--debug show tracebacks of errors and debug messages
-q, --quiet log only warnings and errors
dispatchatlas campaign run¶
usage: dispatchatlas campaign run [-h] [--json] [--debug] [-q] --output DIR
[--workers N] [--backend {sequential,process}]
[--affinity WHERE] [--fresh] [--retry-failed]
[--max-runs N]
CONFIG
Run every planned run that has no record yet. The records go to DIR/<campaign id>/.
Running the command again resumes the campaign. The exit code is 1 when runs failed.
positional arguments:
CONFIG campaign file (.toml or .json)
options:
-h, --help show this help message and exit
--output DIR output directory
--workers N worker processes, or 'auto' for the available processors minus
1 (default: the campaign's setting)
--backend {sequential,process}
run in worker processes, or one at a time in this process
(default: the campaign's setting)
--affinity WHERE processors of the workers: 'inherit' for those of this
process, 'pin' for one of them each, or indices such as 0,2,4
(default: the campaign's setting; --backend sequential implies
'inherit')
--fresh discard every record of the campaign directory first
--retry-failed give every run that failures settled one more attempt
--max-runs N attempt at most N runs in this session
common options:
--json print the result as JSON instead of text
--debug show tracebacks of errors and debug messages
-q, --quiet log only warnings and errors
example:
dispatchatlas campaign run campaign.toml --output results --workers auto
dispatchatlas campaign check¶
usage: dispatchatlas campaign check [-h] [--json] [--debug] [-q] [--replicate-floor N]
[--overrun-tolerance SHARE] [--no-replan]
DIR
Read a campaign directory and report what qualifies its results: whether the stored
campaign still matches its config hash, whether the plan is the plan that campaign
gives, whether every planned run has a valid record or failures that settle it,
whether the budgets were kept, and whether the sessions agree on code and settings.
Each finding is blocking, a limitation, or a note. Nothing is written and nothing
runs. The exit code is 0 when no finding is blocking, 1 when a finding is blocking or
the directory cannot be checked, and 2 for a command line that is not valid.
positional arguments:
DIR campaign directory written by 'campaign run'
options:
-h, --help show this help message and exit
--replicate-floor N report the cells of replicated solvers with fewer than N
completed replicates (default: no floor)
--overrun-tolerance SHARE
share of a time limit by which a completed run may pass it
without counting as over (default: the margin of the engine's
note on a run)
--no-replan do not derive the plan again from the stored campaign, which
loads every problem; the plan file is then compared with the
last session's digest only
common options:
--json print the result as JSON instead of text
--debug show tracebacks of errors and debug messages
-q, --quiet log only warnings and errors
example:
dispatchatlas campaign check results/shop-baselines --replicate-floor 10
dispatchatlas campaign replay¶
usage: dispatchatlas campaign replay [-h] [--json] [--debug] [-q] [--run ID]
[--workers N] [--backend {sequential,process}]
[--affinity WHERE]
DIR
Run the completed runs of a campaign directory again from their recorded seeds and
compare the results with the records. Runs stopped by their wall-clock limit are
skipped. Nothing is written. The exit code is 1 when a run does not reproduce.
positional arguments:
DIR campaign directory written by 'campaign run'
options:
-h, --help show this help message and exit
--run ID replay this run only; repeat for several (default: every
completed run)
--workers N worker processes, or 'auto' for the available processors minus
1 (default: the campaign's setting)
--backend {sequential,process}
run in worker processes, or one at a time in this process
(default: the campaign's setting)
--affinity WHERE processors of the workers: 'inherit' for those of this
process, 'pin' for one of them each, or indices such as 0,2,4
(default: the campaign's setting; --backend sequential implies
'inherit')
common options:
--json print the result as JSON instead of text
--debug show tracebacks of errors and debug messages
-q, --quiet log only warnings and errors
example:
dispatchatlas campaign replay results/shop-baselines --backend sequential
dispatchatlas processes¶
usage: dispatchatlas processes [-h] COMMAND ...
Tools for long runs of calls in pinned child processes: start a command detached from
this shell, watch a runner, list processes with their affinity masks, end processes on
request, and check or repair a rows file.
options:
-h, --help show this help message and exit
commands:
COMMAND
launch Start a command detached from this shell, its output appended to a
log.
watch Print a status line about a runner once per interval; never restart
anything.
list List processes with parent, affinity mask, and start time.
end End processes at once, on request.
rows-check Count the rows of a rows file, or report its torn line.
rows-repair Move the bytes after the last line feed of a rows file aside.
dispatchatlas processes launch¶
usage: dispatchatlas processes launch [-h] [--json] [--debug] [-q] --log LOG
[--cwd CWD] [--env K=V]
...
Start a command detached from this shell, its output appended to a log.
positional arguments:
-- COMMAND the command
options:
-h, --help show this help message and exit
--log LOG file the command's output is appended to
--cwd CWD working directory of the command
--env K=V add an environment variable
common options:
--json print the result as JSON instead of text
--debug show tracebacks of errors and debug messages
-q, --quiet log only warnings and errors
examples:
dispatchatlas processes launch --log run.log --cwd . -- python driver.py --workers 8
dispatchatlas processes watch¶
usage: dispatchatlas processes watch [-h] [--json] [--debug] [-q] --pid PID --rows
ROWS --cpus CPUS [--interval INTERVAL]
[--status STATUS] [--keep]
Print a status line about a runner once per interval; never restart anything.
options:
-h, --help show this help message and exit
--pid PID the runner's process id
--rows ROWS the runner's rows file
--cpus CPUS the run's processors: 0x... or 4-7,12
--interval INTERVAL seconds between polls
--status STATUS file each status line is appended to
--keep keep polling after the runner ends
common options:
--json print the result as JSON instead of text
--debug show tracebacks of errors and debug messages
-q, --quiet log only warnings and errors
examples:
dispatchatlas processes watch --pid 4242 --rows rows.jsonl --cpus 0-7 --status status.log
dispatchatlas processes list¶
usage: dispatchatlas processes list [-h] [--json] [--debug] [-q] [--children-of PID]
[--interpreters-on CPUS]
List processes with parent, affinity mask, and start time.
options:
-h, --help show this help message and exit
--children-of PID only this process's tree
--interpreters-on CPUS
only Python interpreters on these processors
common options:
--json print the result as JSON instead of text
--debug show tracebacks of errors and debug messages
-q, --quiet log only warnings and errors
dispatchatlas processes end¶
usage: dispatchatlas processes end [-h] [--json] [--debug] [-q] PID [PID ...]
End processes at once, on request.
positional arguments:
PID process id
options:
-h, --help show this help message and exit
common options:
--json print the result as JSON instead of text
--debug show tracebacks of errors and debug messages
-q, --quiet log only warnings and errors
dispatchatlas processes rows-check¶
usage: dispatchatlas processes rows-check [-h] [--json] [--debug] [-q] file
Count the rows of a rows file, or report its torn line.
positional arguments:
file the rows file
options:
-h, --help show this help message and exit
common options:
--json print the result as JSON instead of text
--debug show tracebacks of errors and debug messages
-q, --quiet log only warnings and errors
dispatchatlas processes rows-repair¶
usage: dispatchatlas processes rows-repair [-h] [--json] [--debug] [-q] file
Move the bytes after the last line feed of a rows file aside.
positional arguments:
file the rows file
options:
-h, --help show this help message and exit
common options:
--json print the result as JSON instead of text
--debug show tracebacks of errors and debug messages
-q, --quiet log only warnings and errors
dispatchatlas analyze¶
usage: dispatchatlas analyze [-h] [--json] [--debug] [-q] --output DIR [--config FILE]
[--formats LIST] [--no-figures] [--protocol NAME]
[--objective NAME] [--sense {min,max}] [--solver NAME]
[--problem ID] [--alpha A] [--confidence C]
[--correction {none,bonferroni,holm,hochberg,shaffer,finner}]
[--bootstrap-samples N] [--bootstrap-seed N]
[--replicate-floor N] [--focal NAME] [--baseline NAME]
CAMPAIGN_DIR
Analyse the runs of a campaign directory and write the analysis bundle: an index, per-
protocol tables in Markdown, CSV, and LaTeX, figures, the caveats of the analysis, and
a manifest with the settings and every file's SHA-256 digest. Settings come from
--config, then from the options. With --focal, the bundle also holds the table of the
focal solver against each baseline, which is printed too.
positional arguments:
CAMPAIGN_DIR campaign directory
options:
-h, --help show this help message and exit
--output DIR bundle directory
--config FILE JSON or TOML file of analysis settings, such as the manifest
of an earlier bundle, whose settings it repeats
--formats LIST figure formats, comma-separated (default: svg,tex)
--no-figures write the tables and the manifest only
common options:
--json print the result as JSON instead of text
--debug show tracebacks of errors and debug messages
-q, --quiet log only warnings and errors
what to analyse:
--protocol NAME one protocol (default: every one)
--objective NAME objective to compare (default: the primary one)
--sense {min,max} default: the sense the objective declares
--solver NAME analyse this solver; repeat for more
--problem ID analyse this problem; repeat for more
statistics:
--alpha A significance level (default: 0.05)
--confidence C interval coverage (default: 0.95)
--correction {none,bonferroni,holm,hochberg,shaffer,finner}
multiple-comparison correction of the pairwise tests (default:
holm)
--bootstrap-samples N
bootstrap resamples per interval (default: 1000)
--bootstrap-seed N seed of the bootstrap and of the power simulation (default: 0)
--replicate-floor N report cells of stochastic solvers with fewer replicates as a
limitation
comparison with baselines:
--focal NAME solver under study, compared with each baseline
--baseline NAME a baseline; repeat for more (default: every other solver)
examples:
dispatchatlas analyze results/shop-baselines --output analysis/shop-baselines
dispatchatlas analyze results/shop-baselines --output again --config analysis/shop-baselines/manifest.json
dispatchatlas analyze results/shop-baselines --output analysis --focal tabu-search --baseline random-search
dispatchatlas references¶
usage: dispatchatlas references [-h] [--json] [--debug] [-q] [--solver ID]
[--family ID] [--suite ID] [--format {text,bibtex}]
[KEY ...]
Print the bibliography of the methods, benchmarks, and statistics DispatchAtlas
implements: every entry, the entries with the given keys, or the sources of given
solvers, benchmark families, and suites, so you can cite what you use.
positional arguments:
KEY citation key
options:
-h, --help show this help message and exit
--solver ID add the sources of this solver
--family ID add the sources of this benchmark family
--suite ID add the sources of this suite
--format {text,bibtex}
one citation per line, or BibTeX entries (default: text)
common options:
--json print the result as JSON instead of text
--debug show tracebacks of errors and debug messages
-q, --quiet log only warnings and errors
examples:
dispatchatlas references
dispatchatlas references kolisch1996serial --format bibtex
dispatchatlas references --solver tabu-search --suite taillard-flow-shop --format bibtex > sources.bib