Episodes

Snakemake is a Python-based workflow management system for building reproducible and scalable data analysis pipelines Mölder et al., 2021. In this episode, we build a small event-selection workflow and use it to …

In particle physics, we rarely process just one file. A dataset is usually split across many files, we run the same selection on each file, and then we gather the results into a final summary or plot. Snakemake can run …

In the previous episode, we listed the datasets and chunk identifiers in advance. That works well when the workflow already knows what files it should process. Sometimes, however, the file list is only known after an …

In particle physics, we often need software that is awkward to install or keep consistent across different machines. One step may need a modern Python stack, another may need CMSSW, and a third may depend on a specific …

After a workflow runs locally, the next step is often to submit it to an HTCondor cluster. The main idea is simple: the workflow logic should stay the same. In most cases, you should not rewrite rules for HTCondor. …

Getting the Big Picture As your analysis grows from 2 rules to 20, and from 3 samples to 300, it becomes impossible to keep the entire workflow in your head. Snakemake provides built-in tools to “draw” your analysis for …