Neighborhood-scale spatial transcriptomics technologies allow researchers to map molecular profiles across complex multicellular tissue sections, but unresolved sequencing spots frequently record pooled transcripts from multiple cell types rather than isolated single cells. To interpret these composite neighborhoods, computational deconvolution models project observed gene expression mixtures onto mathematical probability simplexes representing relative cell-type fractions. In a study published in Frontiers in Genetics, researchers Rui Yang, Tianfei Ran, Lei Ran, and Zeyu Fu from Army Medical University in Chongqing, China, introduced a standalone implementation called SPGD-TME that surrounds a fixed, training-free baseline estimator with two locked decision operators. The methodology introduces a strict mathematical boundary determining exactly when a malignant cellular coordinate is reliable enough to be reported in published analyses. By establishing an explicit decision contract before downstream interpretation, the system refuses to report unsupported tumor fractions when reference transcriptional programs cannot clearly distinguish cancer cells from neighboring non-malignant tissue compartments.
The foundational component of the framework is a locked reportability gate operating on cosine similarity between normalized gene expression reference columns representing malignant cells and their immediate microenvironmental neighbors. When the calculated column cosine falls strictly below a frozen operational cutoff of 0.80, the algorithm issues a KEEP call and writes a per-spot malignant fraction. Conversely, when malignant and neighbor programs exhibit pronounced collinearity reaching or exceeding the 0.80 threshold, the operator executes an ABSTAIN command. Upon issuing an ABSTAIN decision, the algorithm deliberately withholds the malignant coordinate entirely and holds it as missing rather than reporting an artificial or unstable numerical fraction. The remaining non-malignant cell-type proportions are subsequently renormalized onto a conditional probability simplex. This selective withholding mechanism directly prevents misleading deconvolution outputs in biologically ambiguous spatial regimes, preserving tissue context while alerting investigators that the malignant population cannot be separated from adjacent cellular programs.
Crucially, the reportability gate evaluates cosine similarity strictly on pre-gate reference profiles prior to any library-specific platform corrections or multiplicative Poisson fitting against target tissue spots. The researchers demonstrated that performing platform corrections directly on target spatial spots alters reference gene weighting in a manner that artificially reduces calculated cosine similarities across libraries. For example, applying full platform factor corrections reduced the malignant-to-neighbor cosine from 0.9727 to 0.6895 in openST tissue data, 0.9802 to 0.9447 in breast Xenium sections, and 0.6037 to 0.5609 in CosMx basal-cell carcinoma. In the openST dataset, allowing target-dependent platform adjustments to enter the gate would have erroneously driven a collinear profile across the 0.80 threshold into a false KEEP classification. By anchoring the decision strictly to pre-gate reference profiles before deconvolution fitting begins, the reportability determination remains a fixed, reproducible property of the reference library rather than an artifact of target-dependent tuning.
To evaluate the system under controlled conditions, the investigators deployed the architecture across spatial profiling data from four basal-cell carcinoma patients assayed on the NanoString CosMx Spatial Molecular Imager platform. The experimental cohort incorporated 232,802 individual cells spanning eight shared cell types, from which 216,949 annotated cells were aggregated into 1,194 to 3,261 synthetic mixed spots per patient pair with spot diameters ranging between 42.6 and 45.7 micrometers and pooling medians of 17 to 21 cells per spot. Synthetic spots with fewer than five member cells were excluded, and reference signatures were enforced to meet a pre-agreed builder floor of at least 50 cells per annotated type to prevent low-abundance extrapolations. Under this configuration, the native CosMx basal-cell carcinoma malignant-to-neighbor comparison against normal keratinocytes exhibited a cosine similarity of 0.6037, comfortably clearing the reportability cutoff to trigger an operational KEEP decision.
Alongside the reportability gate, the authors implemented a second locked operator enforcing strict patient-level donor separation to prevent self-training bias during spatial deconvolution validation. By splitting mixed spots and reference libraries by disjoint patient identifiers, no individual patient’s cellular data was ever allowed to train or calibrate the reference model used to deconvolve their own spatial spots. Applying this directed donor transfer across condition-matched donor pairs recovered prominent tissue architectures that were previously verified in single-cell histological fields. Across patients exhibiting ulcerated-nodular tumors, directed transfer from Patient A onto Patient B achieved the tightest overall deconvolution performance in the entire study, recording an overall root-mean-square error of 0.0659, a spot-level Pearson correlation coefficient of 0.9528, a tumor-specific root-mean-square error of 0.1031, and a pseudobulk Jensen-Shannon divergence of 0.0432.
The patient-disjoint transfer simultaneously recovered continuous clinical gradients across an experimental wound healing axis evaluated in Basal-Cell Carcinoma Patients C and D. Along this serial biological axis, average ground-truth tumor cell fractions systematically declined from 0.8425 at baseline condition down to 0.7518 in unwound margins and 0.4665 within active wound beds. In parallel, non-malignant microenvironmental populations expanded, with fibroblast fractions increasing from 0.0881 to 0.2107 and monocyte, macrophage, and dendritic cell populations rising from 0.0070 to 0.1280. When evaluating directed deconvolution transfer of Patient D spots using Patient C references, the computational model precisely captured this morphological shift, yielding an overall root-mean-square error of 0.0906 and a spot-level Pearson correlation coefficient of 0.9037. The parallel rise of stromal and myeloid populations across spots where malignant density fell confirmed that the reported fractions tracked recognizable histological transitions rather than stochastic deconvolution noise.
To examine whether the frozen 0.80 cutoff generalized across diverse oncological contexts, the team audited the gate across five distinct carcinoma datasets profiled on the CosMx platform alongside three external spatial sequencing substrates. Across the CosMx assays, four carcinoma types yielded native cosine values well below the threshold, registering definitive KEEP calls: basal-cell carcinoma at 0.6037, non-small-cell lung carcinoma comparing pooled tumor against fibroblasts at 0.6482, colorectal carcinoma comparing epithelium against fibroblasts at 0.5058, and hepatocellular carcinoma comparing tumor against stellate cells at 0.6486. In sharp contrast, a CosMx pancreatic ductal adenocarcinoma dataset comparing tumor against cancer-associated fibroblasts exhibited a cosine similarity of 0.8616, triggering an immediate ABSTAIN decision on the exact same instrumentation platform that supported KEEP calls in four other tumor cohorts.
External spatial transcriptomics benchmarks from alternate technological platforms similarly produced unequivocal ABSTAIN classifications under the frozen operating cutoff. An openST head and neck squamous cell carcinoma dataset comparing tumor cells against keratin pearls exhibited a high cosine similarity of 0.9727, while a breast carcinoma section profiled on the 10x Genomics Xenium platform comparing invasive tumor against proliferating invasive tumor reached 0.9802. An orthogonal single-cell Chromium Flex reference evaluated against the Xenium dataset yielded a comparable cosine of 0.9770. Collinearity dose-response simulations demonstrated that ungated tumor estimation errors escalated monotonically as synthetic malignant signatures were mathematically interpolated toward neighbor profiles, driving root-mean-square error from 0.1542 to 0.6862 in basal-cell carcinoma and illustrating that the 0.80 gate acts as an effective circuit breaker against collinear breakdown.
Because the pancreatic ductal adenocarcinoma dataset generated an ABSTAIN call on the CosMx system while four other carcinomas passed, the findings indicated that separability is predominantly dictated by biological signature divergence rather than the underlying spatial platform hardware. A variance decomposition performed across 109 eligible non-malignant reference pairs distributed over eight libraries confirmed this distribution, revealing that 71.9 percent of pair-cosine variance resided between distinct cell-type pairs within individual libraries. Differences between libraries within the same platform family accounted for 19.3 percent of the variance, whereas differences between platform families represented only 8.8 percent. Although exact permutation testing across eight libraries remained statistically underpowered with an empirical significance value of p = 0.375, the variance decomposition supported the conclusion that mathematical collinearity represents an intrinsic feature of specific transcriptional phenotypes rather than platform incompatibility.
The computational cost of integrating the locked reportability layer was found to be exceptionally minor relative to the runtime required for baseline composition estimation. Across five consecutive warm execution passes processing a 400-spot openST benchmarking dataset on a multi-core central processing unit without hardware acceleration, complete pipeline builds required an average of 16.103 ± 2.891 seconds. Within this execution profile, fitting the transferable per-gene platform factor accounted for 9.370 ± 2.548 seconds, representing 58.18 percent of total runtime. In stark contrast, calculating the in-memory cosine similarities and executing the reportability gate decision required a mere 0.001 seconds, constituting less than 0.01 percent of the overall build time. This timing profile demonstrated that incorporating formal identifiability checks imposes virtually zero computational overhead on existing spatial transcriptomics workflows.
Despite its mathematical rigor, the authors explicitly outlined important operational constraints and interpretive boundaries surrounding their secondary computational reanalysis. Ground truth in these benchmarks was formalized as cell-count share rather than total transcript mass, meaning that differences in cell volume, absolute RNA content, or probe hybrid capture efficiencies were not captured by the underlying evaluation metrics. Furthermore, the conditional simplex produced following an ABSTAIN decision must be interpreted strictly as relative composition conditioned on tumor withholding, as remaining non-malignant proportions do not represent unbiased estimates of absolute cellular abundance. Emphasizing that their study evaluated computational identifiability rather than patient clinical outcomes, the investigators concluded that selective reporting gates provide an indispensable safeguard against deconvolution overconfidence in complex spatial tissue profiling.
Subject of Research: Algorithmic reportability gating and donor-disjoint validation for spatial transcriptomics tumor microenvironment deconvolution
Article Title: Two locked operators report mixed-spot TME composition on CosMx carcinomas
Article References: Yang, R., Ran, T., Ran, L., & Fu, Z. (2026). Two locked operators report mixed-spot TME composition on CosMx carcinomas. Frontiers in Genetics, 17, Article 1981011. https://doi.org/10.3389/fgene.2026.1981011
Image Credits: AI Generated
DOI: 10.3389/fgene.2026.1981011
Keywords: spatial transcriptomics, tumor microenvironment, cell-type deconvolution, reportability gate, CosMx spatial molecular imager, basal-cell carcinoma, donor transfer, collinearity, cosine similarity, simplex composition, openST, Xenium, non-small-cell lung cancer, pancreatic ductal adenocarcinoma, computational genomics, SPGD-TME
News Source: Nathaniel Bowman. (November 17, 2026). Locked Gating Operators Prevent Misleading Tumor Predictions Across Complex Spatial Tissue Assays. Scienmag.



