Probe and Panel Design for Targeted Spatial Transcriptomics

Choosing which genes to target is as consequential as choosing which spatial transcriptomics platform to use. Targeted spatial transcriptomics probe design determines what a panel can detect and what it will miss. This guide covers how to choose between pre-designed and custom panels, what makes a probe sequence specific, and how to validate a finished panel before committing it to precious tissue.

How targeted spatial transcriptomics differs from whole-transcriptome profiling

Spatial transcriptomics platforms divide into two families based on whether gene selection is required upfront. Sequencing-based platforms capture RNA from barcoded arrays and profile the full transcriptome without preselection, making them the default choice for discovery experiments where the biologically relevant genes are not yet known. Imaging-based platforms, including those based on multiplexed error-robust fluorescence in situ hybridization (MERFISH), sequential fluorescence in situ hybridization (seqFISH), and in situ sequencing (ISS), require probes to be designed against a defined gene list before any experiment can run.

That design requirement reflects a deliberate trade-off rather than a fundamental limitation. Targeted imaging platforms localize every detected transcript to a precise position within, or even inside, a single cell, delivering single-molecule imaging resolution that sequencing-based capture cannot match. The seqFISH+ platform extended this principle by profiling subcellular resolution across genes in mouse brain, imaging 10,000 genes per cell with sub-diffraction-limit accuracy on a standard confocal microscope. Most practical targeted spatial experiments use far smaller panels, where the cost of preselecting genes is more than offset by gains in resolution, per-gene sensitivity, and imaging throughput.

The choice between targeted and whole-transcriptome approaches depends on how well characterized a biological system is before the experiment begins. A discovery-stage study in a poorly annotated tissue is rarely well served by a targeted panel, because the relevant genes may not yet be known and any signal outside the panel is invisible by design. A validation-stage experiment building on prior single-cell or bulk RNA sequencing data is precisely the context in which a targeted panel can be specified with confidence and interrogated at a resolution that captures subcellular transcript localization and cell-type architecture simultaneously.

Choosing between pre-designed and custom spatial panels

Pre-designed panels provide validated probe sequences for gene lists covering well-characterized biology, including pan-tissue cell typing, oncology, and neuroscience programs. Platforms such as Xenium, CosMx, and MERSCOPE each offer ranges of application-specific panels that have been optimized and tested by the vendor across multiple tissue types. These panels lower experimental risk substantially, because sensitivity performance, expected false discovery rates, and cell-type recovery characteristics are documented and benchmarked before a laboratory begins working with them.

Custom panels become necessary when a study's biological question falls outside the scope of available commercial offerings, such as profiling a non-model organism, targeting specific transcript isoforms, or incorporating newly identified markers not yet present in any vendor catalog. Designing a custom panel shifts the probe design and validation burden to the research team, requiring genome annotation resources, bioinformatic design tools, and, ultimately, orthogonal sequencing data to confirm that each probe detects its intended target with acceptable specificity. The additional effort is substantial but unavoidable when no validated pre-designed alternative exists.

Table 1: A comparison of pre-designed and custom targeted spatial transcriptomics panel approaches across key design and operational dimensions.

Core principles of spatial transcriptomics probe design

Every probe sequence in a targeted spatial panel must satisfy a set of physicochemical and bioinformatic criteria before it can be incorporated into an experiment. The core parameters govern hybridization behavior under assay conditions: probe length, GC content, and Tm must all fall within ranges that allow uniform, stable binding across the full panel. Probes that hybridize at substantially different stabilities introduce systematic signal variation that affects detection uniformity across genes and cannot be corrected after data collection. Standard probe design frameworks require that each probe sequence be unique to the intended target gene and absent from all other transcripts in the reference genome, filtering out any candidate that shares substantial sequence homology with off-target transcripts.

The core parameters evaluated during computational probe design include the following.

• GC content, maintained within a platform-specific optimal range to preserve consistent hybridization stability across the full panel
• Melting temperature uniformity, ensuring all probes bind their targets efficiently under the same assay conditions
• Secondary structure avoidance, ruling out sequences capable of forming stable hairpin conformations that compete with target hybridization
• Off-target binding, assessed by sequence alignment against the full reference transcriptome with defined identity and contiguous match-length thresholds
• Probe-probe cross-hybridization, screened to prevent fluorescence signal interference between panel members during the same imaging round

Beyond physicochemical filtering, probe sequences must also be screened for repetitive and low-complexity regions, gene isoforms, and pseudogene sequences that share substantial identity with intended targets. End-to-end probe set selection pipelines now automate many of these steps while simultaneously optimizing the gene list for biological informativeness against a single-cell RNA sequencing (scRNA-seq) reference dataset, selecting genes that capture cell-type variation and within-type expression differences rather than relying solely on canonical marker genes.

How panel size affects plex capacity and sensitivity

Panel size, meaning the total number of genes included, interacts directly with per-gene detection sensitivity. In MERFISH-based platforms, genes are assigned combinatorial binary barcodes decoded across multiple rounds of hybridization and imaging. Each combination of imaging rounds defines a finite barcode space, and expanding a panel beyond the number of valid error-correctable barcodes that space allows requires adding more rounds. More rounds also accumulate background signal from nonspecific probe binding and optical noise, reducing the signal-to-noise ratio for transcripts expressed at low abundance. The error-robust encoding scheme at the core of MERFISH partially offsets this by allowing single-bit detection errors to be identified and corrected during barcode decoding, preserving accuracy even as panel size grows.

ISS-based platforms face related but mechanistically distinct constraints. In ISS, each gene barcode must be sequenced through a defined number of chemistry cycles, so expanding a panel typically requires adding sequencing rounds rather than simply adding new probe sequences. Each additional round increases experiment runtime, reagent consumption, and cumulative mechanical stress on the tissue section, which imposes a practical ceiling on panel size that differs from the sensitivity trade-off in MERFISH. Across both platform types, the consequence is the same: including every potentially interesting gene is rarely the best strategy, because genes that do not contribute to the core biological question consume imaging or sequencing capacity that could otherwise improve detection fidelity for the genes that do.

Optimize Your mRNA Testing Workflows

This brochure outlines a comprehensive suite of analytical services aligned with the latest guidelines and standards from the United States Pharmacopeia (USP).
View How To Guide

Advertisement

Validation strategies for targeted spatial panels

Validating a targeted spatial transcriptomics panel requires confirming that each probe detects its intended target and that aggregate results reflect biologically accurate expression patterns. The most common orthogonal strategy compares per-gene expression levels measured spatially against matched scRNA-seq data from the same tissue type, using correlation analysis to flag genes where the spatial measurement diverges unexpectedly from the single-cell reference. A systematic imaging platform benchmarking study in formalin-fixed, paraffin-embedded (FFPE) tissue found that commercial platforms varied meaningfully in false discovery rates and sensitivity by panel design, and that panels lacking negative control probes were substantially more difficult to evaluate for specificity.

Concordance with sequencing references provides a quantitative validation metric that can be compared across panels, tissue types, and experimental runs. Positive controls embedded within the panel, typically housekeeping genes with predictable, ubiquitous expression, confirm that hybridization chemistry and signal detection functioned throughout the experiment. Negative control probes, targeting sequences absent from the target genome, provide a run-specific estimate of background noise against which all detected signals can be benchmarked. Together, these elements make panel validation an active, quantitative process rather than a retrospective check applied only after a visible failure.

A structured validation sequence for a newly assembled panel proceeds as follows.

1. Run the panel on tissue with well-characterized expression, such as a tissue type previously profiled by scRNA-seq, before committing to novel or irreplaceable samples.
2. Confirm that positive control genes show expected expression distributions consistent with prior reference data for the tissue type.
3. Calculate the false discovery rate from negative control probes or blank barcodes and confirm it falls within the platform's expected range for the panel size.
4. Correlate aggregate per-gene spatial expression levels with matched scRNA-seq or bulk RNA sequencing data and flag genes with unexpectedly low or absent spatial counts.
5. Verify that the panel recovers the expected major cell populations at proportions consistent with prior reference data before proceeding to full-scale experiments.

Designing targeted spatial panels for reliable results

Targeted spatial transcriptomics probe design carries consequences that no downstream computational step can fully undo once data collection is complete. A panel that omits a critical cell-type marker or incorporates a probe with substantial off-target binding introduces systematic error into every subsequent analysis, from cell segmentation and typing to spatial neighborhood statistics. Understanding spatial transcriptomics methods across imaging-based platform families is a prerequisite for probe design decisions, because encoding chemistry, probe architecture, and panel composition constraints differ enough between platforms to make some choices platform-specific rather than universal.

A well-validated panel, whether pre-designed or custom, is the primary asset that makes a targeted spatial experiment interpretable once data collection ends. Assay quality and troubleshooting frameworks address the downstream consequences of probe design choices in greater depth, and both considerations sit within the broader landscape of spatial biology methods and technologies that continues to evolve as panel sizes, probe chemistries, and computational gene-selection tools all mature in parallel.

This content includes text that has been created with the assistance of generative AI and has undergone editorial review before publishing. Technology Networks' AI policy can be found here.