Spatial Transcriptomics Data Analysis: AI Methods for Mapping Gene Expression in Tissue

Spatial transcriptomics data analysis increasingly depends on artificial intelligence (AI) to convert raw, location-tagged gene expression readings into a usable map of tissue biology. Unlike single-cell RNA sequencing (scRNA-seq), spatial transcriptomics preserves the location of each measurement within a tissue section, but recovering cell identities, tissue regions, and spatial patterns from that data is a genuinely hard computational problem. Efforts to confront the challenge sit within the broader shift toward computational life science methods. Machine learning (ML) methods have become the primary tools for solving it at scale.

What spatial transcriptomics data captures

Spatial transcriptomics measures gene expression while recording the physical coordinates of each measurement within a tissue section, something scRNA-seq discards during tissue dissociation. Sequencing-based platforms such as 10x Genomics Visium capture RNA beneath printed spots roughly 55 micrometers (µm) in diameter. As that spot size frequently spans more than one cell, a single measurement can represent a mixture of cell types rather than a single, clean transcriptome. This distinction between spot-based and single-cell resolution platforms is the single most important fact to establish before choosing an analysis method, since it determines whether cell-type deconvolution or direct cell segmentation is the appropriate first computational step.

Imaging-based platforms take a different approach. Multiplexed error-robust fluorescence in situ hybridization (MERFISH) demonstrated highly multiplexed RNA imaging directly within individual cells, accurately imaging 100–1,000 distinct RNA species per cell in its original published demonstration. Vizgen later commercialized this approach directly as its MERSCOPE platform, while other single-cell imaging platforms, including NanoString CosMx, use related but independently developed in situ hybridization chemistries to reach comparable single-cell and subcellular resolution. The trade-off is coverage: single-cell resolution platforms typically profile a few hundred to a few thousand genes rather than the whole transcriptome. This shapes nearly every downstream analytical decision a research team makes, including how much weight MERFISH AI workflows place on the cell and nucleus segmentation step that these imaging platforms depend on to assign transcripts to the correct cell in the first place.

That coverage trade-off is why spatial transcriptomics is often described as sitting downstream of, rather than separate from, the broader push toward AI-driven genomic sequence analysis. Reference cell-type profiles, marker gene panels, and annotation strategies developed for scRNA-seq, as well as genomic sequencing, feed directly into spatial deconvolution and cell-typing models.

Why spatial transcriptomics data analysis is a hard computational problem

A spatial transcriptomics dataset is not simply a gene expression matrix with two extra coordinate columns. Analysis has to jointly reason about expression values, spatial position, and, in many workflows, a paired histology image, which is a fundamentally different problem from clustering a standalone scRNA-seq matrix. Treating spatial coordinates as an afterthought rather than a core input tends to produce clusters and cell-type calls that ignore obvious tissue architecture, such as grouping cells from anatomically distinct regions together simply because their expression profiles happen to look similar in isolation.

Imaging-based platforms introduce their own failure modes. Segmentation errors and transcript spillover, where RNA signal from one cell is misassigned to a neighboring cell during image-based decoding, can distort cell-type calls in dense tissue. A recent evaluation of the Xenium platform documented transcript spillover and signal contamination that measurably affects cell-type specificity. This is why quality control and noise correction are treated as first-class steps in most modern spatial transcriptomics ML pipelines rather than a final cleanup pass.

Comparing or combining multiple tissue sections adds a further layer of difficulty. Sections from different patients, timepoints, or even adjacent slices of the same block rarely align perfectly, so a model built for one sample often needs batch correction or spatially aware alignment before its outputs can be compared across a cohort. Methods that ignore this step risk mistaking a technical artifact of sectioning for a genuine biological difference between samples.

How AI powers Visium data analysis and cell-type mapping

Spot deconvolution addresses the mixed-cell problem directly by estimating the proportions of each spot's transcriptome contributed by each cell type, using cell-type profiles learned from an scRNA-seq reference dataset. Robust cell type decomposition (RCTD) is a widely used example. It was designed to correct for platform-specific technical effects between sequencing technologies, so that mismatches between the reference and target platforms do not masquerade as biological signals.

Cell2location takes a related but distinct approach, using a Bayesian model that performs fine-grained spatial cell mapping by borrowing statistical strength across locations, which achieves higher sensitivity than earlier tools. Its developers demonstrated the method's sensitivity by mapping rare cell subtypes, including populations that made up a small fraction of the reference dataset, to specific locations in mouse brain and human lymph node tissue. The existence of multiple, methodologically distinct deconvolution tools reflects a genuine lack of consensus on the best statistical approach, which is why benchmarking against known tissue biology remains an essential step rather than an optional check.

Deconvolution output is a set of estimated proportions per spot, not a definitive cell-by-cell assignment, and that distinction matters for how results are interpreted. This step is central to Visium data analysis AI workflows specifically because Visium spots mixed cell types, while it is far less relevant to single-cell resolution imaging platforms, where the more pressing computational task is accurate cell segmentation rather than compositional estimation.

Identifying spatial domains using spatial transcriptomics machine learning

Spatial domain identification asks a different question than cell typing. Instead of naming the cell type at a given location, it groups locations into regions that share a coherent expression pattern, often corresponding to an anatomical or functional tissue structure. SpaGCN approaches this with a graph convolutional network that performs histology-guided spatial domain detection, aggregating gene expression across each spot's spatial neighbors before clustering to identify spatially variable genes. Its developers reported that genes identified through this domain-guided approach showed substantially more spatially enriched expression patterns than genes found by competing methods.

Building Aggregates with a Neighborhood Kernel and Spatial Yardstick (BANKSY) unifies cell typing and domain segmentation in a single framework by embedding each cell alongside a summary of its local neighborhood's transcriptome. Its developers reported strong domain segmentation performance against competing approaches on both domain segmentation and cell typing benchmarks. The technique can also be efficiently scaled to datasets containing millions of cells, making it a common default for spatial gene expression AI pipelines that need to run both tasks together.

Cross-modal integration extends this further by treating the paired hematoxylin and eosin (H&E) histology image as additional evidence rather than a static reference picture. Models that combine expression and image features can propagate morphological structure into domain calls and, in some workflows, help predict expression values for genes outside a platform's measured panel. However, predictions of that kind are only as trustworthy as the uncertainty estimates attached to them.

In practice, spatial domain identification tends to matter most in tissues with clear layered or compartmentalized architecture. For example, the cortical layers of the brain or the crypt structures of the intestinal epithelium, where a coherent domain boundary corresponds to a known anatomical feature that a pathologist could independently confirm. That correspondence gives researchers a practical way to sanity-check a model's output before relying on it for downstream biological claims, rather than treating every computed cluster boundary as automatically meaningful.

Key spatial gene expression AI tools and methods

Three open-source tools illustrate how the spatial transcriptomics ML toolkit divides labor across the analysis pipeline (Figure 1). Most working groups combine several of them, alongside deconvolution methods such as RCTD or cell2location and domain identification methods such as SpaGCN, rather than relying on any single package to cover the full workflow.

Figure 1: A five-stage flowchart of the spatial transcriptomics data analysis pipeline, from raw spatial measurements to histology-integrated tissue maps. Credit: AI-generated image created using Google Gemini (2026).

• Squidpy is a Python framework offering scalable spatial omics analysis that brings together omics and image analysis tools for technology-agnostic workflows, including spatial statistics, neighborhood graphs, and image feature extraction.
• BANKSY unifies spatial domain segmentation and cell typing using a neighborhood-aware embedding, and works across sequencing-based, imaging-based, and protein-based spatial datasets.
• Transcript Imputation with Spatial Single-cell Uncertainty Estimation (TISSUE) addresses a problem specific to platforms with limited gene panels. It is a general framework that produces calibrated gene expression predictions with well-defined uncertainty estimates, which reduces false discoveries in downstream differential expression analysis.

How Agentic AI Is Transforming the Modern Lab

This whitepaper examines how agentic AI is reshaping laboratory operations and what this shift means for the future of scientific discovery.
View Whitepaper

Advertisement

A practical spatial transcriptomics data analysis AI workflow generally follows a consistent sequence regardless of platform (Table 1):

1. Start from the raw spot or cell measurements and their spatial coordinates, whichever platform produced them.
2. Perform quality control and segmentation, flagging low-count measurements and, for imaging platforms, likely segmentation artifacts.
3. Apply cell-type deconvolution against an scRNA-seq reference if working with a mixed-cell, spot-based platform such as Visium.
4. Cluster the data into spatial domains using a neighborhood-aware method rather than expression-only clustering.
5. Integrate the paired histology image, when available, through cross-modal integration, and validate the resulting cell-type and domain calls against known marker genes and tissue anatomy before drawing biological conclusions.

Table 1: A qualitative comparison of common spatial transcriptomics platforms and the analytical demands each one creates.

Why AI is now central to spatial transcriptomics data analysis

Spatial transcriptomics data analysis AI methods exist because the underlying biological question, mapping gene expression onto tissue architecture, cannot be answered by treating spatial coordinates as metadata. Deconvolution, spatial domain identification, and cross-modal integration each solve a distinct piece of that problem, and most research groups need several of these approaches working together rather than a single tool. The choice of platform, whether a spot-based sequencing method such as Visium or a single-cell imaging method such as MERFISH or Xenium, largely determines which of these computational steps carries the most weight for a given project.

The tools covered here, including Squidpy, BANKSY, TISSUE, RCTD, cell2location, and SpaGCN, are actively maintained and improving. However, none of them replace the biological judgment needed to interpret a spatial domain or validate a deconvolved cell type against known tissue anatomy. Choosing the right combination for a given platform and tissue type, and validating that combination's output against established markers, remains the most consequential decision in any spatial transcriptomics analysis pipeline.

This content includes text that has been created with the assistance of generative AI and has undergone editorial review before publishing. Technology Networks' AI policy can be found here.