downward.experiment — Fast Downward experiment

class downward.experiment.FastDownwardExperiment(path=None, environment=None, revision_cache=None)[source]

Conduct a Fast Downward experiment.

The most important methods for customizing an experiment are add_algorithm(), add_suite(), add_parser(), add_step() and add_report().

Note

To build the experiment, execute its runs and fetch the results, add the following steps:

>>> exp = FastDownwardExperiment()
>>> exp.add_step("build", exp.build)
>>> exp.add_step("start", exp.start_runs)
>>> exp.add_fetcher(name="fetch")

See lab.experiment.Experiment for an explanation of the path and environment parameters.

revision_cache is the directory for caching Fast Downward revisions. It defaults to <scriptdir>/data/revision-cache. This directory can become very large since each revision uses about 30 MB.

>>> from lab.environments import BaselSlurmEnvironment
>>> env = BaselSlurmEnvironment(email="my.name@unibas.ch")
>>> exp = FastDownwardExperiment(environment=env)

You can add parsers with add_parser(). See Parser for how to write custom parsers and Built-in parsers for the list of built-in parsers. Which parsers you should use depends on the algorithms you’re running. For single-search experiments, we recommend adding the following parsers in this order:

>>> exp.add_parser(exp.EXITCODE_PARSER)
>>> exp.add_parser(exp.TRANSLATOR_PARSER)
>>> exp.add_parser(exp.SINGLE_SEARCH_PARSER)
>>> exp.add_parser(exp.PLANNER_PARSER)
add_algorithm(name, repo, rev, component_options, build_options=None, driver_options=None)[source]

Add a Fast Downward algorithm to the experiment, i.e., a planner configuration in a given repository at a given revision.

name is a string describing the algorithm (e.g. "issue123-lmcut").

repo must be a path to a Fast Downward repository.

rev must be a valid revision in the given repository (e.g., "e9c2370e6", "my-branch", "issue123").

component_options must be a list of strings. By default these options are passed to the search component. Use "--translate-options", "--preprocess-options" or "--search-options" within the component options to override the default for the following options, until overridden again.

If given, build_options must be a list of strings. They will be passed to the build.py script. Options can be build names (e.g., "releasenolp"), build.py options (e.g., "--debug") or options for Make. If build_options is omitted, the "release" version is built.

If given, driver_options must be a list of strings. They will be passed to the fast-downward.py script. See fast-downward.py --help for available options. The list is always prepended with ["--validate", "--overall-time-limit", "30m", "--overall-memory-limit', "3584M"]. Specifying custom limits overrides the default limits.

Example experiment setup:

>>> import os
>>> exp = FastDownwardExperiment()
>>> repo = os.environ["DOWNWARD_REPO"]
>>> rev = "main"

Run iPDB using the latest revision on the main branch:

>>> exp.add_algorithm("ipdb", repo, rev, ["--search", "astar(ipdb())"])

Run blind search in debug mode:

>>> exp.add_algorithm(
...     "blind",
...     repo,
...     rev,
...     ["--search", "astar(blind())"],
...     build_options=["--debug"],
...     driver_options=["--debug"],
... )

Run LAMA-2011 with custom planner time limit:

>>> exp.add_algorithm(
...     "lama",
...     repo,
...     rev,
...     [],
...     driver_options=[
...         "--alias",
...         "seq-saq-lama-2011",
...         "--overall-time-limit",
...         "5m",
...     ],
... )
add_suite(benchmarks_dir, suite)[source]

Add PDDL or SAS+ benchmarks to the experiment.

benchmarks_dir must be a path to a benchmark directory. It must contain domain directories, which in turn hold PDDL or SAS+ files (ending with “.pddl” or “.sas”).

suite must be a list of domain or domain:task names.

>>> benchmarks_dir = os.environ["DOWNWARD_BENCHMARKS"]
>>> exp = FastDownwardExperiment()
>>> exp.add_suite(benchmarks_dir, ["depot", "gripper"])
>>> exp.add_suite(benchmarks_dir, ["gripper:prob01.pddl"])
>>> exp.add_suite(benchmarks_dir, ["rubiks-cube:p01.sas"])

One source for benchmarks is https://github.com/aibasel/downward-benchmarks. After cloning the repo, you can generate suites with the suites.py script. We recommend using the suite optimal_strips for optimal STRIPS planners and satisficing for satisficing planners:

# Create standard optimal planning suite. $
path/to/downward-benchmarks/suites.py optimal_strips ['airport',
..., 'zenotravel']

Then you can copy the generated list into your experiment script:

>>> exp.add_suite(benchmarks_dir, ["airport", "zenotravel"])

Built-in parsers

The following constants are paths to built-in parsers that can be passed to exp.add_parser(). The “Used attributes” and “Parsed attributes” lists describe the dependencies between the parsers.

FastDownwardExperiment.EXITCODE_PARSER

Parsed attributes: “error”, “planner_exit_code”, “unsolvable”.

FastDownwardExperiment.TRANSLATOR_PARSER

Parsed attributes: “translator_peak_memory”, “translator_time_done”, etc.

FastDownwardExperiment.SINGLE_SEARCH_PARSER

Parsed attributes: “coverage”, “memory”, “total_time”, etc.

FastDownwardExperiment.ANYTIME_SEARCH_PARSER

Parsed attributes: “cost”, “cost:all”, “coverage”.

FastDownwardExperiment.PLANNER_PARSER

Used attributes: “memory”, “total_time”, “translator_peak_memory”, “translator_time_done”.

Parsed attributes: “node”, “planner_memory”, “planner_time”, “planner_wall_clock_time”, “score_planner_memory”, “score_planner_time”.