sprocket run

Individual tasks and workflows can be run with the sprocket run subcommand. We outline a few of the important considerations below, but we encourage you to run sprocket run --help to see all available arguments and options.

Execution backends

See the section on execution backends to learn more about configuring Sprocket to execute tasks in different environments.

Targets

The task or workflow to run can be provided explicitly with the --target argument.

sprocket run --target main example.wdl

Whether or not this argument is required is based on whether inputs are provided to Sprocket from which the target can be inferred (e.g., providing an input of main.is_pirate implies a target of main). Conversely, if you supply a --target, you don't have to prefix your command line inputs with the target's fully qualified name.

Sprocket will indicate when it cannot infer the target.

Inputs

Inputs to a Sprocket run are provided as arguments passed after the WDL document name is provided. Each input can be specified as either

• a key value pair (e.g., main.is_pirate=true)
• a JSON file containing inputs (e.g., a hello_defaults.json file where the contents are { "main.is_pirate": true })
• a YAML file containing inputs (e.g. a hello_defaults.yaml file where the contents are main.is_pirate: true)

Inputs are incrementally applied, meaning that inputs specified later override inputs specified earlier. This enables you to do something like the following to use a set of default parameters and iterate through sample names in Bash rather than create many individual input files.

sprocket run example.wdl hello_defaults.json main.name="Ari"

An example

Using the the WDL document from the guided tour, we can specify the name parameter as a key-value pair on the command line.

sprocket run example.wdl --target main name="World"

After a few seconds, this job runs successfully with the following outputs.

{
  "main.messages": [
    "Hello, World!",
    "Hallo, World!",
    "Hej, World!"
  ]
}

If you wanted to override some of the defaults for the workflow, you could do so by defining the input in a hello_overrides.json file:

{
  "main.greetings": [
    "Good morning",
    "Good afternoon",
    "Good evening"
  ],
  "main.is_pirate": true
}

Then providing that file in the set of inputs to the workflow.

sprocket run example.wdl hello_overrides.json main.name="Sprocket"

This produces the following output.

{
  "main.messages": [
    "Good morning, Sprocket!",
    "Good afternoon, Sprocket!",
    "Good evening, Sprocket!",
    "Ahoy, Sprocket!"
  ]
}

Output directory

By default, sprocket run writes all execution artifacts to ./out. This can be changed with the -o, --output-dir flag.

sprocket run example.wdl --target main name="World" -o /path/to/output

Individual runs are stored at <output_dir>/runs/<target>/<timestamp>/, and a _latest symlink is maintained for each target pointing to its most recent run. The output directory also contains a SQLite provenance database (sprocket.db) that tracks all executions.

The --index-on flag enables output indexing, creating symlinks under <output_dir>/index/<output_name>/ for efficient lookup of runs by output values.

sprocket run hello.wdl -t hello --index-on greeting

For full details on the output directory structure, provenance database, and output indexing, see the Provenance Tracking documentation.