Execution Backends
Sprocket evaluates a workflow locally and decomposes it into tasks. Sprocket dispatches these tasks to a configured execution backend responsible for starting, monitoring, and responding to task lifecycle events.
Sprocket currently supports the following execution backends:
- A local Docker execution backend.
- A remote Task Execution Service backend.
- An LSF + Apptainer HPC backend (experimental).
- A Slurm + Apptainer HPC backend (experimental).
Backend configuration
A sprocket.toml configuration file may contain settings for zero or more execution backends, but currently Sprocket uses only one specified backend during a workflow evaluation.
The execution backend is specified in the run section:
[run]
backend = "<name>"The name corresponds to an entry in the run.backends table. A backend setting is required if there is more than one backend or if the backends table does not contain a default entry.
An example of configuring more than one backend:
[run]
backend = "server1"
[run.backends.server1]
type = "tes"
# ...
[run.backends.server2]
type = "tes"
# ...If no execution backend is configured in sprocket.toml, the Docker backend is used.
Container Protocol Support
Different execution backends support different container image protocols. The table below summarizes which protocols are supported by each backend.
| Protocol | Docker | TES | LSF + Apptainer | Slurm + Apptainer |
|---|---|---|---|---|
docker:// | ✓ | ✓ | ✓ | ✓ |
library:// | ✓ | ✓ | ✓ | |
oras:// | ✓ | ✓ | ✓ | |
file:// (local SIF) | ✓ | ✓ |
Protocol Details
docker://— OCI/Docker registry images (e.g.,docker://ubuntu:22.04or simplyubuntu:22.04). This is the default protocol when no scheme is specified.library://— Sylabs Cloud Library images (e.g.,library://sylabs/default/alpine:3.18). Supported by Apptainer-based backends and TES servers that support Sylabs Library references.oras://— OCI Registry As Storage images (e.g.,oras://ghcr.io/org/image:tag). Supported by Apptainer-based backends and TES servers that support ORAS.file://— Local Apptainer/Singularity.siffiles (e.g.,file:///path/to/image.sif). Only supported by Apptainer-based backends where the file is accessible from the execution environment.