Reproducible bioinformatics workflows with Snakemake and Nextflow

A workflow makes file dependencies explicit so a runner can determine what must run, what can run in parallel, and what outputs are missing. This walkthrough compares Snakemake and Nextflow, then builds a small file-copy workflow you can adapt before connecting real sequencing tools.

R remains the main language for analysis steps in this collection. A workflow can call a reviewed R script with Rscript --vanilla and render an R Markdown report as a later step. The definitions below must still use the manager's native syntax: Snakemake's Python-based rule language and Nextflow's Groovy-based DSL. They are not alternative R/Python implementations of a biological analysis.

Start with the file contract

Write down each step as an input-to-output contract before choosing a workflow manager. For a two-step analysis, the contract might be: raw reads plus a reference produce an alignment; the alignment plus an annotation produce a count table. Record the exact filenames, sample identifiers, software versions, parameters, and expected outputs. A workflow cannot correct a mismatched sample sheet, an incomplete reference, or an inappropriate biological method.

Shell scripts remain useful for short, one-off tasks. As a project gains samples or steps, implicit assumptions become harder to track: a command may depend on the current directory, an earlier output may be partial, or a later command may start before its input is ready. Workflow managers represent dependencies and schedule jobs from those dependencies. They can resume work when the declared files and software environment permit it, but a file's existence alone does not prove it is complete or scientifically valid.

A small Snakemake workflow

Snakemake describes rules with inputs and outputs. The requested target determines which rules are needed; wildcard patterns let one rule describe the same operation for multiple sample names. This synthetic example copies one plain-text fixture to a results directory so you can check the mechanics without installing a bioinformatics tool.

SAMPLES = ["demo"]

rule all:
    input:
        expand("results/{sample}.txt", sample=SAMPLES)

rule copy_fixture:
    input:
        "data/{sample}.txt"
    output:
        "results/{sample}.txt"
    shell:
        "cp {input:q} {output:q}"

Save this as Snakefile, create data/demo.txt containing a short line of text, and run snakemake --cores 1 from that directory. The default target is rule all; its expanded input names the desired result, and Snakemake matches that path to copy_fixture. The {sample} wildcard is inferred from the target and substituted into the input path. The :q formatter quotes paths passed to the shell command. Inspect results/demo.txt and compare it with the input. This is a software fixture, not evidence about a sequencing analysis.

Dry runs and failures

Use snakemake -n before a larger run to inspect the jobs Snakemake plans to execute without running them. A missing input should stop rule planning or execution with an informative error. If a result is unexpectedly considered current, inspect the target, input paths, timestamps, and whether the rule fully declares the files it creates. A dry run checks the planned graph; it does not validate the scientific suitability of the commands.

How Nextflow represents the same work

Nextflow describes processes and connects their inputs and outputs through channels. A channel carries values or file paths; a process declares what it consumes, the command to run, and what it emits. This dataflow model differs in emphasis from Snakemake's file-target graph, although both tools can express multi-step, parallel workflows. The following small DSL2 example copies the same kind of plain-text fixture; it demonstrates process input/output staging and publication, not a biological analysis.

Run the matching text-fixture workflow

Use Bash, Nextflow, and a Java release supported by your installed Nextflow version; this text fixture was checked with Nextflow 26.10.3 and Java 21 on Ubuntu Linux. Check java -version and nextflow info first; follow the official installation guide if either command is unavailable. Starting from your working directory, create a separate project and save the following Groovy script there as main.nf:

mkdir -p nextflow-demo/data
cd nextflow-demo
printf 'fixture text for workflow practice\n' > data/demo.txt

params.input = 'data/demo.txt'

process COPY_FIXTURE {
    publishDir 'results', mode: 'copy'

    input:
    path source

    output:
    path 'copied.txt'

    script:
    """
    cp "$source" copied.txt
    """
}

workflow {
    source_ch = channel.fromPath(params.input, checkIfExists: true)
    COPY_FIXTURE(source_ch)
}

Run nextflow run main.nf. In the process, source is the staged path for the file found by channel.fromPath; cp "$source" copied.txt writes the declared output in the task directory. The publishDir 'results', mode: 'copy' directive copies that declared output to results/copied.txt in the launch directory.

Try one failure: run nextflow run main.nf --input data/missing.txt. It should fail before the process runs and name the missing path. This command does not rename or delete the original input; rerun the original command to use data/demo.txt again.

Nextflow records task cache entries on each run, but reuses them only when you explicitly pass -resume. Use nextflow run main.nf -resume to request reuse when the task is eligible; keep the associated cache and work directories. See the official caching and resuming guide and publication directive reference.

For a real read-mapping workflow, specify whether reads are single-end or paired-end and keep each pair associated with its sample identifier. The source example's single-read BWA command did not correctly demonstrate paired-end handling, and its channel/process declarations were not a verified runnable contract, so that code has been removed rather than presented as executable. When converting an existing command, first run it on one sample outside the workflow, then represent the same inputs, resources, and output files in the process declaration.

Choose by workflow needs, not labels

Neither choice is universally easier or more reproducible. Version control the workflow and configuration, record input provenance and software versions, and retain run logs and output checksums where appropriate. Pinning software helps another person reconstruct the computational environment; it does not guarantee identical results across hardware, nondeterministic tools, or changed external reference data.

Running an nf-core pipeline safely

nf-core pipelines are versioned community workflows with their own input schemas and parameters. Do not copy an invocation without checking the pipeline's current documentation. For nf-core/rnaseq, the official usage page documents a samplesheet and requires reference inputs such as a genome FASTA and annotation GTF for the relevant configuration. It also recommends testing the setup with the pipeline's test profile before running real data. The test profile is a setup check, not a substitute for validating your study design, sample metadata, or reference build.

For production, select a specific pipeline release, read its parameter and output documentation, choose a container or package profile supported by your environment, and confirm that the profile is permitted on your cluster or cloud account. Avoid using a mutable “latest” workflow or an unreviewed parameter file for a long-lived analysis. Keep credentials and private data out of the workflow repository and logs.

Practice and self-check

Further reading

• Snakemake general-use tutorial and Snakemake command-line documentation
• Nextflow: your first script and process documentation
• nf-core/rnaseq usage and outputs