Episode

Containers

In particle physics, we often need software that is awkward to install or keep consistent across different machines. One step may need a modern Python stack, another may need CMSSW, and a third may depend on a specific ROOT build.

Snakemake lets a rule describe not only its inputs and outputs, but also the software environment it should run in. That makes workflows more portable and more reproducible, because the rule can carry its environment with it.

Why Containers Matter

Without containers, your workflow depends on whatever happens to be installed on the machine where it runs. With containers, the workflow can say exactly which environment a given step should use.

This is especially useful in HEP because:

• the same workflow may run on a laptop, on a shared system, or on batch nodes
• different rules may genuinely need different software stacks
• the analysis logic should stay the same even when the execution environment changes

A First Containerised Rule

We can reuse the plot.py script from the previous episode and run the gather step inside a Python container.

DATASETS = ["DYJets", "TTbar", "Data"]
CHUNKS = ["0", "1", "2"]


rule all:
    input:
        "plots/event_counts.txt"


rule select_events:
    input:
        "input/{dataset}/{dataset}.{chunk}.txt"
    output:
        "selected/{dataset}/{dataset}.{chunk}.txt"
    shell:
        """
        mkdir -p "selected/{wildcards.dataset}"
        grep "Selected" "{input}" > "{output}" || test $? -eq 1
        """


rule make_plot:
    input:
        expand(
            "selected/{dataset}/{dataset}.{chunk}.txt",
            dataset=DATASETS,
            chunk=CHUNKS,
        )
    output:
        "plots/event_counts.txt"
    container:
        "docker://python:3.11-slim"
    script:
        "plot.py"

The workflow logic is unchanged: make_plot still gathers the selected files and writes the summary. The new part is the container: directive, which tells Snakemake which image to use for that rule.

Running the Workflow with Apptainer

To make Snakemake execute containerised rules, use:

pixi run snakemake --cores 4 --sdm apptainer

export APPTAINER_CACHEDIR="${TMPDIR}/.apptainer-cache"
mkdir -p "${APPTAINER_CACHEDIR}"

export APPTAINER_CACHEDIR="/tmp/${USER}/.apptainer-cache"
mkdir -p "${APPTAINER_CACHEDIR}"

[tasks]
snakemake-apptainer = { cmd = "snakemake --cores 4 --sdm apptainer", env = { APPTAINER_CACHEDIR = "$TMPDIR/.apptainer-cache" } }

pixi run snakemake-apptainer

What Happens Behind the Scenes?

When Snakemake sees a rule with a container: directive and you run with Apptainer enabled, it will:

1. resolve the requested image
2. pull it if needed and cache it locally
3. execute the job inside that container
4. make the workflow files available to the job

From the rule author’s point of view, this is the important part: the rule still declares input, output, and how to run the step. The container just defines the software environment for that step.

Why Per-Rule Containers Are Useful

Different parts of a workflow may need different environments. Snakemake allows that naturally:

rule old_software_step:
    output:
        "intermediate.root"
    container:
        "docker://my-old-root-image:latest"
    shell:
        "run_old_code > {output}"


rule modern_python_step:
    input:
        "intermediate.root"
    output:
        "plots/final_summary.txt"
    container:
        "docker://python:3.11-slim"
    script:
        "plot.py"

That is a major advantage over trying to manage the whole workflow inside one shared login environment.

Practical Notes

The idea of containers is the same everywhere, but some execution details depend on the system where you run the workflow.

• On a local machine, you need Apptainer installed on the system.
• On shared systems, Apptainer may already be available.
• External paths may need additional bind mounts.
• Exact execution details can depend on the local site configuration.

A Small Reality Check