Decentralized Federated Learning Cleans Noisy Labels Using Peer Model Majority Voting

Computer scientists Guan Huang and Tao Shu have developed a decentralized federated learning framework named DFLMV that enables distributed edge devices to overcome corrupted training annotations without relying on central servers or compromising private local data. In decentralized networks, autonomous vehicles, aerial drones, and distributed sensors frequently collaborate to train artificial intelligence models directly on local hardware without transferring sensitive raw records to external repositories. However, real-world data collection routinely suffers from mislabeled samples caused by human carelessness or limited domain expertise, which severely degrades model convergence, accuracy, and operational robustness. Rather than relying on conventional data-filtering pipelines that require central coordination or direct inspection of training samples, the newly introduced approach coordinates peer-to-peer consensus directly across neighbor models. By allowing participating edge clients to evaluate local samples using models received from neighboring peers, the system reliably identifies and corrects inaccurate labels before executing downstream model fine-tuning across the decentralized computational collective.

Accurate data annotation remains an indispensable prerequisite for training deep neural networks, yet imperfect annotations are virtually inevitable across large-scale distributed deployments. Empirical studies have documented extensive label inaccuracies within widely used computer vision benchmarks, including an MIT investigation finding incorrect labels on approximately ten percent of QuickDraw, nearly six percent of the ImageNet test set, and almost six percent of CIFAR-100. When machine learning shifts from centralized data centers to federated settings, this vulnerability intensifies because participating edge clients possess widely disparate annotation skills, attention spans, and specialized domain knowledge. In decentralized federated learning environments lacking a central coordinator, such imbalances produce severe disparities in local dataset quality across network nodes. Unchecked label noise damages model generalization, impairs internal feature interpretability, and degrades overall prediction reliability. Resolving these adverse effects across resource-constrained edge platforms operating over loose peer connections represents an urgent challenge for real-world distributed artificial intelligence infrastructures.

Prior attempts to mitigate noisy training data in distributed computing predominantly depended on centralized federated architectures or traditional centralized learning environments. Centralized methods typically attempt to estimate global noise transition matrices, select clean sample subsets through multi-pass validation, or train auxiliary noise filters across pooled client data. However, these techniques fundamentally conflict with decentralized federated learning architectures, which operate exclusively through many-to-many peer communications across resource-constrained edge nodes. Direct client data inspection violates core decentralized privacy policies, intensive peer-to-peer clean sample filtering incurs prohibitive computational and synchronization bottlenecks, and server-based consensus models collapse under sporadic network connectivity. Furthermore, simple majority voting applied directly to shared data labels fails when noise levels escalate. If voters share binary labels with an independent error rate of fifty percent, conventional data voting produces an identical fifty percent error rate, and error rates worsen when noise exceeds fifty percent, actively degrading latent label reliability instead of improving it.

To resolve these structural limitations, the authors structured DFLMV into a three-stage training pipeline that operates entirely without central servers or supplementary clean validation datasets. During the initial training phase, each participating client trains its local neural network model on its uncorrected local dataset using standard stochastic gradient descent across designated local epochs to minimize local empirical risk. In each communication round, every client broadcasts its updated parameter vector to a small neighborhood of randomly selected communication peers, receiving parameter vectors in return. The client then aggregates these incoming weights using the decentralized Federated Averaging algorithm before initiating subsequent local training cycles. This decentralized exchange continues for a predetermined number of communication rounds, designated as the label correction threshold, or until training loss ceases to decline across ten consecutive rounds. Crucially, this initial phase allows local models to absorb broader global structural representations while maintaining compliance with strict peer-to-peer communication boundaries.

Upon reaching the specified communication threshold, the framework transitions into its second phase, where clients perform a one-time label correction operation across their local training datasets. Instead of broadcasting private training samples or exchanging raw data annotations, each client receives model parameter weights from its currently available peer devices. The client then passes every local training sample through each received peer model, generating an array of independent label predictions for each individual data point. Because distinct edge clients possess different subsets of training data and noise realizations, their local models develop distinct classification perspectives. The hosting client tallies these predictions and applies an unweighted plurality majority voting rule to identify the consensus label for each local sample. The client immediately substitutes this consensus label for the original potentially corrupted annotation, constructing an improved local training set with substantially reduced label noise while ensuring that raw features and individual training samples never leave the local device.

In the final phase of the framework, each edge client resumes model training by fine-tuning its parameters on the newly corrected local dataset across remaining communication rounds. Clients utilize the model weights preserved from the conclusion of the initial training phase as their fine-tuning baseline rather than restarting model training from scratch. Throughout these remaining rounds, clients continue executing local gradient descent epochs and exchanging aggregated weights with their randomly selected peer groups under the standard Federated Averaging routine. The researchers established that DFLMV accomplishes this systematic correction while introducing minimal communication and computational overhead compared to traditional decentralized federated workflows. The overall communication overhead remains bounded at the standard order of transmitted model parameters, as peer devices only exchange weight vectors. Meanwhile, the supplementary computation overhead is confined to the label correction phase, scaling strictly linearly with the number of local training samples being evaluated by neighbor models.

The study provides formal mathematical proofs establishing the theoretical advantages of voting on model inferences rather than voting on raw data samples. The authors derived a universal upper bound on generalization error for decentralized federated learning algorithms employing cross-entropy loss functions under arbitrary label noise distributions. Additionally, they formulated an upper bound on majority voting error in multi-class classification using modified Bessel functions of the first kind, assuming non-colluding neighbors with identical voting distributions. Their theoretical analysis demonstrated that the upper bound on label correction error decreases monotonically as the number of neighboring voting models increases, asymptotically approaching zero in infinite neighbor limits. Furthermore, higher peer model quality directly tightens this error bound. Crucially, the mathematical findings proved that decentralized models fine-tuned on datasets corrected via model-level majority voting achieve strictly lower generalization error bounds than models trained exclusively on corrupted raw datasets, guaranteeing systematic accuracy improvements across arbitrary noise ratios.

To evaluate DFLMV across realistic deployment conditions, the researchers conducted extensive empirical benchmarking across seven diverse image classification datasets on an array of sixteen Tesla P100 graphics processors. The experimental framework evaluated synthetic benchmarks, including MNIST, Fashion-MNIST, and CIFAR-10, across twelve rigorous configurations combining independent and non-independent data partitions with symmetric, pairflip, and asymmetric label noise models at ratios of ten, thirty, and fifty percent. Data heterogeneity among the ten participating clients was generated using a Dirichlet distribution with a concentration parameter of one and a half. In addition to synthetic corruption, the team tested DFLMV on four real-world noisy benchmark datasets featuring naturally occurring human annotation errors: CIFAR-10N, CIFAR-100N, Clothing1M, and ANIMAL-10N under non-independent partitions without artificial noise injection. Network connectivity constraints were systematically enforced across all evaluations by restricting each client to exchanging parameter weights with only five randomly selected peer nodes per communication round.

The empirical results demonstrated that DFLMV consistently outperformed standard decentralized Federated Averaging, a modified peer-to-peer baseline called PENS, and decentralized adaptations of three recent server-based methods: FedCorr, FedDiv, and CLC. In challenging synthetic evaluations with fifty percent label noise under non-independent data and noise distributions, DFLMV increased test accuracy by 20.95 percentage points over decentralized Federated Averaging and 23.68 percentage points over PENS. When compared against the decentralized adaptations of FedCorr, FedDiv, and CLC across nine non-independent synthetic configurations at fifty percent noise, DFLMV achieved superior classification accuracy in every setting, outperforming the strongest adapted baseline by up to 8.92 percentage points. The authors observed that the magnitude of improvement was particularly pronounced in heterogeneous environments, where diverse local model perspectives dilute erroneous peer predictions across multiple incorrect classes, allowing correct predictions to form dominant pluralities that successfully rectify local dataset corruption and enhance final model performance.

Sensitivity analyses regarding the timing of the label correction phase revealed that allocating approximately two hundred communication rounds to initial decentralized training yielded optimal test accuracy on CIFAR-10 under severe non-independent noise. Setting the correction round below two hundred prematured the voting stage before peer models attained sufficient classification capability, while extending the threshold beyond two hundred and fifty rounds starved the retraining phase of necessary fine-tuning iterations. The authors emphasized that their theoretical and experimental investigations focused specifically on non-malicious annotation errors resulting from annotator carelessness or lack of specialized knowledge, explicitly excluding adversarial data poisoning or deliberate client collusion. Additionally, the current implementation relies strictly on unweighted plurality voting across peer predictions. Future research will explore weighted voting mechanisms and probabilistic consensus formulations to further advance label error mitigation across decentralized federated systems operating in mission-critical environments such as autonomous transportation, healthcare telemetry, and satellite communications.

Subject of Research: Decentralized federated learning over noisy training labels using model majority voting

Article Title: Decentralized federated learning over noisy labels: a majority voting method

Article References: Huang, G., & Shu, T. (2026). Decentralized federated learning over noisy labels: a majority voting method. Neural Computing and Applications, 38(20), Article 850. https://doi.org/10.1007/s00521-026-12582-2

Image Credits: AI Generated

DOI: 10.1007/s00521-026-12582-2

Keywords: Federated Learning, Decentralized Machine Learning, Label Noise, Majority Voting, Edge Computing, Neural Networks, Data Privacy, Empirical Risk Minimization, Decentralized, federated, learning, over