Machine Learning Model Predicts Secondary Testosterone Deficiency Using Routine Clinical Data

A computational framework combining a relational autoencoder with an optimized gradient boosting classifier has achieved high diagnostic precision in detecting secondary testosterone deficiency from routine clinical and biochemical parameters. Published in Discover Computing by researchers P. John William and E. Ilavarasan, the study introduces an artificial intelligence pipeline designed to overcome the diagnostic challenges of male hypogonadism. Secondary testosterone deficiency represents an endocrine disorder that is particularly prevalent among aging men, yet it frequently remains underdiagnosed during clinical evaluations. The condition encompasses a spectrum of physical, metabolic, and psychological disturbances, including fatigue, depression, reduced libido, and an elevated likelihood of cardiovascular complications. Conventional diagnostic pathways typically depend on isolated hormone measurements and subjective assessments of symptoms, which can prove insufficient for consistent detection. To address these operational shortcomings, the researchers engineered a specialized two-level architecture that extracts compressed latent representations while preserving structural correlations between patient records, enabling accurate identification from inexpensive examinations.

Male hypogonadism and testosterone depletion involve complex physiological mechanisms that evolve over a person’s lifespan. Testosterone is the principal circulating androgen in males, produced predominantly by Leydig cells in the testicles under the tight regulation of the hypothalamic-pituitary-gonadal axis, with minor hormonal contributions from the adrenal glands. Throughout prenatal development and puberty, testosterone directs external genital masculinity, musculoskeletal growth, and secondary sexual characteristics. In adult males, it sustains erectile function, libido, spermatogenesis, bone density, muscle mass, cognitive function, and cardiovascular health. However, systemic production decreases at an estimated rate between 0.4 percent and 2.0 percent annually after the age of 30, with men in their seventh decade demonstrating average plasma testosterone concentrations roughly 35 percent lower than younger cohorts. Approximately 30 percent of men between the ages of 40 and 70 are affected by deficiency disorders, a demographic proportion projected to climb alongside global gains in life expectancy.

Diagnosing secondary testosterone deficiency remains an intricate clinical challenge because its presenting symptoms overlap substantially with normal aging and various chronic metabolic disorders. Epidemiological surveys reflect significant regional variance in low testosterone prevalence, reporting biochemical rates between 24 and 39 percent among aging men in the United States, 8 to 20 percent across Europe, and 17 to 33 percent in Asia and South America. Symptomatic presentations are documented in roughly 6 percent of middle-aged men. The clinical literature highlights a bidirectional link between testosterone deficiency and metabolic syndrome, with low circulating levels closely connected to hypertension, visceral adiposity, lipid abnormalities, and impaired glycemic control. Despite the widespread nature of the condition, therapeutic management remains inconsistent: surveys indicate that up to 25 percent of patients initiating testosterone replacement therapies receive no initial hormone testing, while nearly half are never monitored post-treatment, and many appropriate candidates are denied therapy due to unverified clinician concerns regarding cardiovascular disease and prostate cancer.

Existing computational efforts to predict testosterone deficiency have encountered persistent technical hurdles related to class imbalance, noise, and nonlinear biomedical interactions. While several earlier investigations evaluated classical algorithms such as random forests, support vector machines, AdaBoost, and traditional gradient boosting architectures, many struggled to generalize effectively across heterogeneous patient cohorts or required cost-prohibitive specialized diagnostic inputs. Simpler machine learning techniques often fail to accommodate the intricate nonlinear relationships inherent to clinical metabolic profiles, whereas standard deep neural networks frequently demand massive annotated datasets and heavy computational overhead. Furthermore, conventional autoencoders tend to optimize only input reconstruction errors, overlooking the valuable structural relationships that link individual patient profiles across high-dimensional feature spaces. These performance deficits highlighted the clinical need for an interpretable, lightweight, and scalable computational model capable of generating robust predictions from routine health metrics.

To construct their predictive framework, the researchers obtained a clinical cohort comprising 3,395 male patients aged 40 to 85 years from a urological medical institution located in Feira de Santana, Bahia, Brazil. Collected by the UroS group at the Universidade Estadual de Feira de Santana under ethical approval number 3,057,301, the dataset incorporated 1,886 individuals diagnosed with Type 2 diabetes. Men presenting with primary hypogonadism or those actively undergoing hormone replacement therapies were systematically excluded from the experimental cohort. The analytical dataset focused strictly on six easily accessible, cost-effective predictor variables gathered during routine outpatient examinations: patient age in years, serum triglycerides in milligrams per deciliter, waist circumference in centimeters, high-density lipoprotein cholesterol in milligrams per deciliter, hypertension status as a binary indicator, and diabetes status as a binary indicator. Total serum testosterone served as the clinical target variable, with secondary testosterone deficiency formally defined as a concentration of 300 nanograms per deciliter or lower.

Data preprocessing established a structured foundation to prevent methodological bias, eliminate scale discrepancies, and prevent data leakage between evaluation sets. The raw records were first cleaned by removing uninformative identifier columns and eliminating entries containing missing or partial clinical values. To ensure rigorous evaluation, the researchers divided the 3,395 patient records into an 80 percent training partition and an isolated 20 percent testing cohort. Min-Max normalization was applied to rescale all continuous attributes into a standardized numerical range between zero and one, with normalization parameters calculated strictly from the training partition and subsequently applied to the held-out test data. Because clinical records frequently exhibit skewed class proportions, the investigators implemented random oversampling exclusively on the training subset, duplicating minority secondary deficiency cases to achieve parity. The held-out test cohort remained completely unadjusted, preserving its natural class distribution to guarantee an unbiased, realistic appraisal of model performance on unseen clinical cases.

The first core technical tier of the modeling pipeline utilized a Relational Autoencoder designed to capture essential nonlinear feature dependencies while preserving patient-to-patient similarity structures. Unlike traditional autoencoders that minimize reconstruction error alone, the relational network incorporated a dual-objective mathematical formulation balancing data reconstruction loss with relational preservation loss. Operating across a symmetric 6–128–64–32–16–32–64–128–6 neural layer layout, the encoder mapped the six preprocessed clinical indicators down into a compact 16-dimensional latent representation. A rectifier activation filtering function was integrated into the relational objective, systematically pruning weak, inconsequential associations while prioritizing prominent feature interactions. Trained via stochastic gradient descent until reaching loss convergence thresholds, the autoencoder compressed the clinical feature space into informative latent matrices. This transformation extracted salient representations of underlying metabolic profiles, significantly reducing computational dimensionality while capturing physiological nuances that individual clinical variables could not expose when analyzed in isolation.

The secondary phase of the analytical architecture employed an optimized LightGBM classifier to perform final diagnostic classification using the 16-dimensional relational latent vectors. LightGBM, a high-performance open-source gradient boosting decision tree framework developed by Microsoft, was chosen for its rapid execution speeds, low memory footprint, and native suitability for complex clinical datasets. The classifier leveraged histogram optimization techniques, which discretize continuous variable values into integer bins during decision tree construction, thereby dramatically decreasing redundant mathematical calculations. Furthermore, the gradient boosting pipeline implemented Gradient-based One-Side Sampling to retain training instances with large gradients while randomly subsampling those with smaller gradients, alongside Exclusive Feature Bundling to combine mutually exclusive sparse attributes without informational loss. This integrated combination allowed the gradient boosted trees to efficiently delineate complicated decision boundaries across the relational latent space without incurring heavy computational burdens during training.

Experimental evaluations executed using Python 3.7.9, TensorFlow, and Keras on a standard personal computer configuration revealed outstanding diagnostic accuracy on the testing dataset. When applied to the unaltered held-out cohort of 679 patients, the hybrid architecture correctly classified 378 individuals as healthy and 278 patients as having secondary testosterone deficiency, incurring only 12 false positives and 11 false negatives. This translated to an overall testing accuracy of 96.61 percent with a 95 percent confidence interval spanning 94.97 to 97.73 percent. The model achieved a precision of 95.86 percent, a recall of 96.19 percent, a specificity of 96.92 percent, and an overall F1-score of 96.03 percent. Ablation experiments demonstrated that the relational latent features contributed substantially to predictive performance: a standard artificial neural network reached 91.84 percent accuracy, standalone LightGBM achieved 94.72 percent, and a conventional autoencoder coupled with LightGBM reached 95.63 percent, confirming that relationship-preserving representations provided unique predictive value.

Comparative analyses against earlier machine learning benchmarks demonstrated the superiority of the relational autoencoder approach over existing models in the hypogonadism literature. Previously documented frameworks such as XGBoost, AdaBoost, standard Random Forests, and support vector machines had reported diagnostic accuracies ranging from 77.80 percent to 93.60 percent, often accompanied by degraded precision or unmeasured specificity. Despite the promising experimental results, the authors outlined several key methodological limitations that contextualize the findings. The model was trained and evaluated on cross-sectional data from a single regional urological clinic, lacking external validation against patient populations from independent medical centers. Furthermore, the clinical dataset omitted longitudinal hormone tracking and critical lifestyle variables such as physical exercise, alcohol intake, and tobacco usage. The authors emphasized that while the software provides an efficient, scalable decision-support tool for flagging patients requiring endocrine workups, algorithmic predictions are intended to assist clinical decision-making rather than replace laboratory confirmation or specialist medical evaluation.

Subject of Research: Machine learning prediction of secondary testosterone deficiency from clinical data

Article Title: Prediction of secondary testosterone deficiency using a hybrid robust auto encoder and light gradient boosting model

Article References: William, P. J., & Ilavarasan, E. (2026). Prediction of secondary testosterone deficiency using a hybrid robust auto encoder and light gradient boosting model. Discover Computing, 29(1), Article 856. https://doi.org/10.1007/s10791-026-10773-y

Image Credits: AI Generated

DOI: 10.1007/s10791-026-10773-y

Keywords: testosterone deficiency, secondary hypogonadism, machine learning, LightGBM, relational autoencoders, clinical diagnostics, computational healthcare, endocrinology, Prediction, secondary, testosterone, deficiency