Demos — public-data runs per technology¶

This page shows how to run SpaceBlocks end-to-end on public data for each supported technology, and documents how to prepare your own inputs for mode: decoupled.

Repository's synthetic data

The tiny case in .test/ exists only to exercise the DAG in CI, and to render the workflow's tube map on the Snakemake catalog.

Technical notes¶

To keep the tutorials simple and lightweight, we use 1 sample from a public dataset and divide it into 3 artificial samples, keeping only 500 highly-variable genes (HVGs).

We provide default config files, pre-annotated regions (GeoJSON files), clusters and cluster-to-cell type equivalences for the example public datasets to ensure reproducibility.

The exact environment versions used during the generation of these tutorials can be found under demos/lock.envs.

Each of the examples below can run under 8Gb of RAM and 4 cores in <40 min under our slurm HPC system:

Our HPC specifications

7 standard compute nodes (HP Apollo 2000 Gen10+)
2 × AMD EPYC 7513 CPUs (32 cores, 2.8 GHz each)
256 GB RAM per node

1 high-memory compute node (HP Apollo 2000 Gen10+)
2 × AMD EPYC 7513 CPUs (32 cores, 2.8 GHz each)
1 TB RAM

Visium HD uncoupled mode example (mouse brain)¶

SpaceBlocks is built to analyse single-cell resolution Spatial Transcriptomics data, so Visium HD data needs to be preprocessed via bin2cell, ENACT or, as in the Visium HD HeadBlock, Space Ranger >= v4.0.1 (internally implementing StarDist segmentation).

Space Ranger version matters

We use a Space Ranger ≥ 4.0.1 Visium HD dataset (one that ships segmented_outputs/). The older Space Ranger 3.x Mouse Brain release has no StarDist segmentation, so format_visiumhd.py would find no cell table.

For the example here presented, you may download the Visium HD dataset from the 10x Genomics web, or via terminal using curl or wget. This dataset has been processed with Space Ranger v4.0.1.

Visium HD demo dataset size and SpaceBlocks mode

This demo runs in mode: decoupled and the size of the dataset is ~15GB.

# at the workflow dir, build the environment
cd SpaceBlocks/
conda env create --file=workflow/envs/visiumhd.yaml
# activate it
conda activate visiumhd
# move to the reproducibility dir and place the downloaded outs under data/<SAMPLE>/
# move to the reproducibility dir and download the three files under data/<SAMPLE>/
cd demos/visiumhd
mkdir -p data/Visium_HD_Mouse_Brain && cd data/Visium_HD_Mouse_Brain
BASE=https://cf.10xgenomics.com/samples/spatial-exp/4.0.1/Visium_HD_Mouse_Brain
curl -O $BASE/Visium_HD_Mouse_Brain_segmented_outputs.tar.gz
curl -O $BASE/Visium_HD_Mouse_Brain_barcode_mappings.parquet
curl -O $BASE/Visium_HD_Mouse_Brain_binned_outputs.tar.gz
tar -xzf Visium_HD_Mouse_Brain_segmented_outputs.tar.gz    # -> segmented_outputs/
tar -xzf Visium_HD_Mouse_Brain_binned_outputs.tar.gz       # -> binned_outputs/
cd ../..
# build the contracts for the CoreBlocks
# use an interactive job with 16Gb and 4 cores on a slurm cluster (salloc --ntasks=1 --cpus-per-task=4 --mem 16G -t 01:00:00)
python format_visiumhd.py            # -> contracts/<sample>.h5ad + core_samples.tsv

We provide demo_vhd.geojson for this dataset, as an example QuPath export on the hires image (see QuPath annotation tutorial).

Xenium 5K uncoupled mode example (human melanoma)¶

You may download the Xenium dataset from the 10x Genomics web, or via terminal using curl or wget.

Xenium 5K demo dataset size and SpaceBlocks mode

This demo runs in mode: decoupled and the size of the dataset is ~11GB.

# at the workflow dir, build the environment
cd SpaceBlocks/
conda env create --file=workflow/envs/xenium5k.yaml
# activate it
conda activate xenium5k
# move to the reproduciblity dir
cd demos/xenium5k
# download the Xenium bundle into data/<SAMPLE>/
mkdir -p data && cd data
curl -O https://cf.10xgenomics.com/samples/xenium/3.0.0/Xenium_Prime_Human_Skin_FFPE/Xenium_Prime_Human_Skin_FFPE_outs.zip
unzip Xenium_Prime_Human_Skin_FFPE_outs.zip -d Xenium_Prime_Human_Skin_FFPE
cd ..
# build the contracts
# use an interactive job with 16Gb and 4 cores if using a slurm cluster (salloc --ntasks=1 --cpus-per-task=4 --mem 16G -t 01:00:00)
python format_xenium.py              # -> contracts/<roi>.h5ad + core_samples.tsv

Notice that the Xenium GeoJSON is annotated in pixels, while cells are in microns. The format_xenium.py script detects the scale mismatch and rescales the polygons automatically.

Importantly, format_xenium.py embeds a greyscale composite of the morphology_focus channels in uns["spatial"], so spatial plots are drawn over the tissue instead of on a bare scatter. Set HIRES_LEVEL in the script to trade resolution for file size.

Atera HeadBlock example (human breast cancer, alpha)¶

Alpha support

Atera is expected to ship in the second half of 2026. The dataset below is a public preview, generated with development software, and 10x state its output format will change at commercial release. This is a HeadBlock test rather than a polished demo, and it is expected to need revision once the final format lands.

Unlike the two examples above, this one runs a HeadBlock rather than mode: decoupled: the contract h5ad is built by prepare_input_ate from the raw bundle, so it exercises the whole head chain. The dataset is 18,028 genes over 170,057 cells (CC BY 4.0).

Atera demo dataset size and SpaceBlocks mode

This demo runs in mode: atera and the size of the dataset is ~190GB.

# at the workflow dir, build the environment
cd SpaceBlocks/
conda env create --file=workflow/envs/atera.yaml
conda activate atera
# download into a scratch dir with ~120 GB free: the bundle alone is ~55 GB zipped
mkdir -p /path/to/atera && cd /path/to/atera
BASE=https://s3-us-west-2.amazonaws.com/10x.files/samples/atera/dev/WTA_Preview_FFPE_Breast_Cancer
curl -O $BASE/WTA_Preview_FFPE_Breast_Cancer_outs.zip
unzip WTA_Preview_FFPE_Breast_Cancer_outs.zip -d outs
# optional but recommended: the registered H&E and its alignment
SUP=https://cf.10xgenomics.com/samples/atera/dev/WTA_Preview_FFPE_Breast_Cancer
curl -O $SUP/WTA_Preview_FFPE_Breast_Cancer_he_image.ome.tif
curl -O $SUP/WTA_Preview_FFPE_Breast_Cancer_he_alignment.csv
curl -O $SUP/WTA_Preview_FFPE_Breast_Cancer_keypoints.csv

Then point atera.atera_dir (and, if you fetched them, the three he_* keys) in demos/atera/atera_config.yaml at that directory, and run the head:

snakemake qupath_images qc_sweep_all    --sdm conda   # OPTIONAL regions: annotate a QuPath TIFF (see below), export the GeoJSON
snakemake run_preprocessing             --sdm conda

The shipped config annotates cells directly from 10x's own per-cell calls (annotation_types: [external_annotation]), so the run needs no manual annotation step and postprocessing is reproducible from the supplemental CSVs alone. Convert them with the helper that ships next to the config:

# before the run: 10x's own cell calls -> external-annotation metadata
python demos/atera/format_atera.py metadata \
    /path/to/atera/WTA_Preview_FFPE_Breast_Cancer_cell_groups.csv \
    --sample WTA_Preview_FFPE_Breast_Cancer --outdir demos/atera/external_metadata

snakemake run_postprocessing --sdm conda

The helper also prints the annotation_colors block using 10x's own display colours, already pasted into the shipped config.

The usual SpaceBlocks route — a human mapping Leiden clusters to cell types in cluster_annotations.tsv is to be implemented post-alpha. Note that leiden_analysis stays on to showcase 10x Genomics marker table demos/atera/snakemake_cell_markers.tsv, which is the supplemental gene_groups.csv pivoted to one column per group.

Region annotation is optional here (the demo annotates cells from 10x's own calls). If you do want regions, qupath_images exports two annotatable TIFFs per sample, and prepare_input_ate reads whichever GeoJSON you export named after the image stem ({sample}_<image>.geojson), preferring them in this order:

{sample}_he_background.tiff — the registered H&E resampled onto the morphology grid (also the embedded contract background). Recommended: it sits on the cells' coordinate grid, so regions map back with a single scalar (no affine). Export as {sample}_he_background.geojson.
{sample}_morphology.tiff — the fluorescence composite (the fallback when no H&E is configured). Export as {sample}_morphology.geojson.

Importantly, the registered H&E is not the raw whole-slide scan, which covers a second tissue section that Atera never imaged.

The bundle holds a single sample, so run_postprocessing is not meaningful/complete here because integration, pseudobulk DE and composition comparisons all need replicates. The shipped config therefore trims those options; see demos/xenium5k/xenium5k_config.yaml for a config that exercises them.

MERSCOPE HeadBlock example (mouse brain)¶

Like the Atera demo, the MERSCOPE demo runs its HeadBlock the contract h5ad is built by prepare_input_mer straight from a raw Vizgen region, so it exercises the whole MERSCOPE head chain (generate_qupath_mer → prepare_input_mer). The head reads the two Vizgen cell CSVs directly and reuses the 5k.yaml environment (no intermediate zarr, no new environment).

MERSCOPE demo dataset size and SpaceBlocks mode

This demo runs in mode: merscope and the size of the dataset is ~25GB.

We use the open-access Vizgen MERFISH Mouse Brain Receptor Map (Vizgen Data Release V1.0), Slice 1 / Replicate 1 — 483 genes, ~78k cells, a full coronal slice at single-cell resolution. Vizgen distributes this dataset through its Data Release Program landing page rather than an open CDN, so there is no single curl URL: request/download the files from info.vizgen.com/mouse-brain-map (the underlying Google Cloud bucket is requester-pays)

# from SpaceBlocks/, set the data Google Cloud dirs
BASE="gs://public-datasets-vizgen-merfish/datasets/mouse_brain_map/BrainReceptorShowcase/Slice1/Replicate1"
DEST=demos/merscope/data/BrainReceptorShowcase_S1R1
mkdir -p "$DEST/images"
# download the data for the demo using gcloud
gcloud storage ls "$BASE/images/"
gcloud storage cp "$BASE/cell_by_gene_S1R1.csv"   "$DEST/"
gcloud storage cp "$BASE/cell_metadata_S1R1.csv"  "$DEST/"
gcloud storage cp "$BASE/images/micron_to_mosaic_pixel_transform.csv" "$DEST/images/"
gcloud storage cp "$BASE/images/mosaic_DAPI_z3.tif"  "$DEST/images/"
gcloud storage cp "$BASE/images/mosaic_PolyT_z3.tif" "$DEST/images/"

Make sure that you do not run into problems with google-crc32c. After donwload, the data should look like this:

demos/merscope/data/BrainReceptorShowcase_S1R1/
├── datasets_..._cell_by_gene_S1R1.csv        # raw transcripts-per-cell matrix
├── datasets_..._cell_metadata_S1R1.csv       # centroids (center_x/y, µm) + QC
└── images/
    ├── datasets_..._micron_to_mosaic_pixel_transform.csv
    ├── ..._mosaic_DAPI_z3.tif                 # nuclear stain  (always present)
    └── ..._mosaic_PolyT_z3.tif               # total-RNA stain (always present)

The mouse brain release predates the Cell Boundary Stain Kit, so it ships DAPI + PolyT only. The MERSCOPE HeadBlock composites whatever channels it finds and skips absent ones, so no config change is needed.

If your download names its mosaics with a z-plane other than z3, set merscope.z_index to a plane that exists (the head errors clearly if the requested channel/z is missing).

# at the workflow dir, build the environment (shared with the Xenium head)
cd SpaceBlocks/
conda env create --file=workflow/envs/xenium5k.yaml

# point merscope.merscope_dir in demos/merscope/merscope_config.yaml at the region above,
# then select the config in workflow/Snakefile (see "Configure and run" below) and run:
snakemake qc_sweep_all qupath_images       --sdm conda   # OPTIONAL qc_sweep and regions: annotate the composite TIFF,
                                                         # export demos/merscope/geojson/<sample>_morphology.geojson
snakemake run_preprocessing                --sdm conda   # builds the contract via the head, then the CoreBlock

generate_qupath_mer writes <sample>_morphology.tiff (the annotation image), a grey background, and a <sample>_morphology_scalefactors.json that records the mosaic-affine mapping once; prepare_input_mer reads it back so cell centroids, the embedded background and any QuPath regions all share one grid (see design for the contract shape).

Region annotation is optional. Without a GeoJSON, region_annotation is Unlabeled.

The shipped config holds a single region, so postprocessing (integration, pseudobulk DE, composition comparisons) is not meaningful and has been trimmed from the merscope_config.yaml, exactly as in the Atera demo. To exercise the full core, add Slice 1 Replicate 2 and Replicate 3 as extra rows in demos/merscope/core_samples.tsv (they are genuine biological replicates) and re-enable the postprocessing options (see demos/xenium5k/xenium5k_config.yaml for a config that uses them).

Configure and run the examples¶

A ready-to-run config ships next to each example: demos/xenium5k/xenium5k_config.yaml, demos/visiumhd/visiumhd_config.yaml, demos/atera/atera_config.yaml and demos/merscope/merscope_config.yaml. Optionally, you may use an external reference for the ingest rule, we provide external links to set in the example config files.

To use one, comment the default configfile: line in workflow/Snakefile and uncomment the matching example line (all are already present):

# configfile: "config/config.yaml"
# configfile: demos/visiumhd/visiumhd_config.yaml      # alternative for visiumhd
# configfile: "demos/atera/atera_config.yaml"          # alpha: Atera HeadBlock test
# configfile: "demos/merscope/merscope_config.yaml"    # MERSCOPE HeadBlock test
configfile: "demos/xenium5k/xenium5k_config.yaml"

DO NOT INCLUDE several config files

Make sure to only have one uncommented config file in the Snakefile. Snakemake allows the Snakefile to reference multiple config files, so having multiple config files could lead to unexpected results.

Then launch the core:

snakemake run_preprocessing qc_sweep_all --sdm conda
snakemake run_postprocessing             --sdm conda   # integration + DE across the 3 synthetic samples

What preconfigured files exercise¶

Config files¶

The shipped demos/xenium5k/xenium5k_config.yaml is set up to demonstrate the optional rules too:

Subcompartment re-clustering. subcompartments groups cell type labels and re-clusters them (subcluster rule, included in run_posprocessing).
Gene / signature exploration. gene_queries_demo.tsv holds a small gene set and a gene to showcase the run_exploration CoreBlock capabilities.
Automatic annotation (optional). ingest_ref and ingest_ref_label_key are empty by default, they allow the use of external references for automatic annotation (which may guide manual annotation). We provide examples in the config files to test this feature during qc_sweep_all (showing which assigned cell types are removed during QC) and run_preprocessing. Automatic annotation will also be shown in the sample_report.pdf during run_postprocessing.

TSV cluster annotation files¶

Manual assignation of Leiden clusters into cell types (via config/cluster_annotations.tsv) is the default requirement to run the postprocessing and exploration CoreBlocks in SpaceBlocks.

Example cell type annotation files are provided as cluster_annotations_{technology}_demo.tsv to show how the TSV should be filled.

Preparing inputs for decoupled mode¶

The config mode: decoupled skips the HeadBlocks entirely and runs the CoreBlocks on contract h5ads you provide. The two scripts above are worked examples of how to produce them (with artificial samples).

To generate valid contract AnnData objects (saved as h5ads), follow the contract specification; the rules in short are:

1 — Shape. Each h5ad must satisfy the contract: X = raw integer counts, obsm["spatial"] = per-cell (x, y), and obs carrying sample (and, ideally, cell_id and region_annotation). cell_id is optional but strongly recommended, and must equal obs_names to prevent unexpected behaviour — downstream joins key on it, and validate_input warns when it differs. An embedded uns["spatial"] image is optional, but also strongly recommended. validate_input checks all of this before anything runs.

2 — Naming & layout. One file per sample, laid out as:

<contract_dir>/<sample>.h5ad

where each <sample> is a row in config/core_samples.tsv. The sample names in the sheet, the folder names, and the obs["sample"] values must all match. The example scripts (see above) guarantee this. You can find the TSV examples in demos/<technology>/contracts/core_samples.tsv.

3 — Config. Point the run at them:

mode: "decoupled"
contract_dir: "demos/visiumhd/contracts"   # holds <sample>.h5ad
core_samples: "demos/visiumhd/contracts/core_samples.tsv"
# plus post_processing_outdir, logdir, geojson_path (see Configuration)

Region annotation in decoupled mode is baked into the contract (as region_annotation) when you build it — there is no qupath_images target and no GeoJSON join later.