Privacy-Preserving Federated Learning for Single-Cell Foundation Models

Researchers have introduced Clifti-GPT, a privacy-preserving federated framework designed to fine-tune and apply foundation models to clinical single-cell RNA sequencing data without centralizing sensitive patient information. Published in BioData Mining, the study presents a software solution that addresses the strict data protection policies governing heterogeneous clinical cohorts, where the centralization of raw data is often prohibited by institutional governance and regulatory requirements such as the General Data Protection Regulation (GDPR). The framework enables collaborative model training and transferable inference across decentralized repositories, allowing institutions to leverage the power of large-scale pre-trained models while keeping raw data, clinical-level statistics, and model weights local to each site.

The core of the Clifti-GPT architecture is built upon the scGPT foundation model, which is pre-trained on the extensive CELLxGENE single-cell RNA sequencing dataset encompassing over 33 million cells. This foundation model utilizes self-attention transformer architectures to capture complex biological patterns across diverse cell types, tissues, and species. However, applying such models to specific clinical tasks typically requires fine-tuning on large, heterogeneous cohorts that are often siloed across different hospitals. Clifti-GPT bridges this gap by distributing the pre-trained scGPT model to participating sites, where local fine-tuning is performed, and model updates are aggregated using secure multi-party computation (SMPC).

Unlike standard federated learning, which may still be vulnerable to gradient-leakage or model-inversion attacks, Clifti-GPT employs additive secret sharing as a cryptographic mechanism. In this process, client updates are split into secret shares that are distributed to independent computational parties. These parties aggregate the shares without reconstructing any client’s gradients in plaintext, ensuring that intermediate parameters remain protected under a semi-honest threat model. This approach distinguishes between privacy-preserving computation processes and results, mitigating the risk of sensitive information leaking from intermediate updates or embeddings during the training phase.

Beyond fine-tuning, the framework introduces a concept of transferable inference for zero-shot applications, such as reference mapping for cell type annotation. In this mode, predictions are performed collaboratively across decentralized sites without transferring raw data or high-dimensional embeddings. Instead, only complementary statistics derived from comparisons of secret-shared embeddings, such as nearest-neighbor distances and voting counts, are exchanged and aggregated under SMPC. This design allows institutions to benefit from cross-site knowledge for annotation tasks while adhering to institutional governance constraints, as the information entropy of the exchanged results is inherently minimized.

The study evaluated the effectiveness of Clifti-GPT across six public single-cell RNA sequencing datasets, including Multiple Sclerosis, Human Pancreas, Lung-Kim, Cell Line, Myeloid, and Covid-19 cohorts. These datasets were partitioned into reference and query sets to simulate realistic federated scenarios with substantial biological and technical non-IID heterogeneity. The researchers compared the performance of the federated models against centralized baselines, where all data are pooled, and local models, where each client trains independently on its limited private data. The evaluation metrics included accuracy, precision, recall, and macro-F1 scores for cell type classification and reference mapping.

Results demonstrated that Clifti-GPT achieves performance within 4% of centralized scGPT baselines in accuracy, precision, recall, and macro-F1 for cell type classification across the evaluated datasets. In several scenarios, such as the Cell Line and Lung-Kim datasets, the federated models matched or even slightly outperformed the centralized models on specific metrics. The framework also showed rapid convergence in terms of communication rounds, reaching 99% of centralized performance on cell type classification in at most two federated rounds on two evaluated datasets. This communication efficiency is a critical advantage for practical deployment, as it reduces the computational and network overhead associated with multi-institutional collaboration.

To assess scalability, the researchers distributed the Myeloid dataset across varying numbers of clients, creating scenarios with 5, 10, 20, and 30 clients. In the most heterogeneous Top30 scenario, the federated model maintained performance within a worst-case margin of 3% across all metrics, demonstrating robustness to extreme data fragmentation. The study also examined the impact of batch effects, which are technical variations that can confound single-cell data analysis. Using the Covid-19 dataset, the researchers found that batch effects had a stronger impact on federated analysis than on centralized training. However, after applying batch effect correction using the scGen tool, the privacy-preserving federated model closed the performance gap, achieving an accuracy of 0.93, which closely matched the centralized baseline.

The computational overhead of the privacy-preserving workflows was also profiled using a GPU-accelerated CrypTen SMPC framework. Secure aggregation for federated binning and fine-tuning was found to be lightweight, completing within tens of milliseconds for models with one million trainable parameters. In contrast, secure k-nearest neighbor (KNN) for reference mapping emerged as the dominant system bottleneck, with runtime overheads of 11,000 to 70,000 times compared to plaintext execution. This latency is attributed to the intensive network communication required for encrypted distance computations and secure comparison-based reductions. Despite this bottleneck, the study suggests that a decoupled architecture, where clinical sites delegate expensive cryptographic operations to dedicated high-performance computational nodes, could alleviate the burden on data-contributing hospitals.

The authors emphasize that the experiments conducted are controlled simulations of multi-client deployment rather than deployed institutional trials. While the framework uses genuine biological and batch structure to approximate distributional heterogeneity, it does not fully capture all operational features of a real multi-institutional deployment, such as differences in sequencing protocols, quality-control thresholds, and metadata standards. The privacy claims are limited to the semi-honest SMPC threat model, where participants follow the prescribed protocol but may inspect available information. The framework does not claim robustness to malicious clients, collusion, or active protocol deviations, which would require complementary defenses such as Byzantine-robust aggregation or malicious-secure SMPC.

Furthermore, the study highlights that additive secret sharing does not intentionally add privacy noise, preserving the intended computation up to implementation-level numerical effects. This design choice prioritizes utility while reducing direct exposure of protected intermediate quantities. However, output-privacy concerns, such as model inversion or membership inference from the final global model or predictions, remain separate from the SMPC guarantee of computation-process privacy. The researchers note that a comprehensive end-to-end pipeline for real-world deployment will require extending the framework to include additional preprocessing components, such as privacy-preserving gene-space alignment and batch-effect correction, which introduce unique cryptographic and statistical challenges.

Clifti-GPT represents a significant step toward integrating foundation models into distributed clinical workflows, offering a blueprint for adopting next-generation models in a privacy-preserving and communication-efficient manner. By enabling robust performance in settings with extreme non-IID distributions, the framework addresses the practical requirements of data governance and GDPR-relevant design principles. As single-cell data continues to expand, such privacy-preserving federated approaches will be essential for unlocking the full potential of foundation models in biomedical research while respecting institutional boundaries and patient privacy.

Subject of Research: Clifti-GPT: privacy-preserving federated fine-tuning and transferable inference of foundation models on clinical single-cell data

Article Title: Clifti-GPT: privacy-preserving federated fine-tuning and transferable inference of foundation models on clinical single-cell data

Article References: Bakhtiari, M., Elkjaer, M. L., Can, A. O., Theis, F., Oubounyt, M., & Baumbach, J. (2026). Clifti-GPT: privacy-preserving federated fine-tuning and transferable inference of foundation models on clinical single-cell data. BioData Mining, 19(1), Article 58. https://doi.org/10.1186/s13040-026-00582-w

Image Credits: AI Generated

DOI: 10.1186/s13040-026-00582-w

Keywords: Clifti-GPT, privacy-preserving, federated, fine-tuning, transferable, inference, foundation, models, clinical, single-cell, data, scientific research