Latch offers native Snakemake support. This repository contains a minimal Snakemake workflow and Latch metadata to demonstrate this functionality.

The pipeline runs a short two-step scatter-gather over a list of samples provided via an input samplesheet.

It also includes a conditional rule that will only execute if a LatchFile does not exist at an expected location. The output of this rule is copied back to this expected location, to cache the results for future workflow executions.

This example workflow is intended to test and demonstrate the following behaviors:

• Load a sample manifest during Latch's JIT,
• Use input functions to link metadata provided in the samplesheet to actual file paths on Latch,
• Conditional rule execution, with cached results

https://wiki.latch.bio/docs/snakemake/quickstart
https://wiki.latch.bio/docs/snakemake/tutorial

Clone the repo, create and activate the conda environment, and run the pipeline.

git clone [email protected]:msto/latch_native_snakemake.git
cd latch_native_snakemake

mamba env create -f environment.yml
mamba activate latch_native_snakemake

snakemake --configfile config.yml -j 1

Within the working directory, and with the latch_native_snakemake environment activated, register the workflow.

latch register . --snakefile Snakefile

Following the instructions in the quickstart and tutorial, I generated metadata and a Dockerfile with the Latch CLI.

latch generate-metadata config.yml --snakemake
latch dockerfile . --snakemake

I made the following modifications to the default file contents:

• latch_metadata/parameters.py: I set download=True for the metadata_tsv parameter, so the list of input samples can be loaded during Latch's JIT (per the docs)

  latch_native_snakemake/latch_metadata/parameters.py

  Line 32 in f55851f

  download=True,

• latch_metadata/__init__.py: I set appropriate values for the metadata

  latch_native_snakemake/latch_metadata/__init__.py

  Lines 7 to 11 in f55851f

  	output_dir=LatchDir("latch://23203.account/latch_native_snakemake/output"),
  	display_name="Native Snakemake Proof-of-Concept",
  	author=LatchAuthor(
  	name="Matt Stone",
  	),

• Dockerfile: I update the conda environment created by the Dockerfile to include the additional dependencies specified in my environment.yml

  latch_native_snakemake/Dockerfile

  Line 55 in f55851f

  RUN mamba env update --file /root/environment.yml --name latch_native_snakemake

• .dockerignore: I set up an allowlist limited to the workflow and Latch files.

  latch_native_snakemake/.dockerignore

  Lines 1 to 15 in f55851f

  	# Ignore everything (all local data, results, analysis, and notebooks)
  	*
  	# Include the dockerfile
  	!Dockerfile
  	# Include the workflow, config, environment, and toolkit
  	!Snakefile
  	!config.yml
  	!environment.yml
  	# Include Latch metadata
  	!latch_metadata
  	!.latch
  	!version

Other notes:

• The Snakefile may not include a configfile directive. Specify the config at the command line for local use.
• The usual pattern of reading an input samplesheet must be modified so the read is wrapped in a function call. After JIT, each task attempts to run all Python code present in the file outside rule definitions. https://wiki.latch.bio/docs/snakemake/cloud#make-sure-shared-code-doesnt-rely-on-input-files