Machine Learning Reveals Key Knowledge Factors Linked to Tuberculosis Treatment Adherence

In an exploratory cross-sectional study evaluating outpatients receiving tuberculosis care at a district-level medical center, researchers deployed an explainable machine learning pipeline to detect behavioral patterns and knowledge deficits correlated with medication non-adherence. Tuberculosis continues to pose a formidable global public health challenge, ranking as the second leading infectious cause of death worldwide behind COVID-19, with millions of incident cases and fatalities recorded every year. Vietnam remains categorized among the thirty nations bearing the highest tuberculosis burdens internationally, facing entrenched clinical and programmatic hurdles. Successful management relies on patient adherence to complex multi-drug therapeutic regimens lasting a minimum of six months, where irregular dosing or premature interruption frequently leads to treatment failure, disease relapse, extended community transmission, and the development of multidrug-resistant tuberculosis strains.

Understanding the multi-faceted determinants that govern therapeutic compliance is essential for preventing default, as adherence behaviors are shaped by intricate combinations of clinical, socio-demographic, and psychological elements. Previous investigations have shown that personal comprehension of the disease, societal stigma, and adverse pharmaceutical reactions often undermine ongoing medication persistence. Concurrently, environmental barriers such as health service quality, financial distress, and the absence of domestic support systems can profoundly hinder consistent compliance. While traditional public health research has predominantly utilized conventional logistic regression models to identify vulnerable patients, these classic statistical approaches depend heavily on linear assumptions and are often compromised by class imbalance in observational datasets, where compliant patients typically constitute the substantial majority over defaulting individuals.

To capture non-linear behavioral interactions while resolving the opacity that often impedes the clinical translation of advanced computational algorithms, the investigators constructed an analytical architecture coupling eXtreme Gradient Boosting with SHapley Additive exPlanations. The clinical dataset was assembled from direct structured interviews conducted between May 2025 and December 2025 among active tuberculosis outpatients aged 18 years and older participating in programmatic treatment. Enrolling a total sample of 379 patients via consecutive non-probability sampling, the preliminary inquiry initially tracked 46 variables capturing clinical characteristics, socio-demographic indicators, and disease literacy across the cohort, providing a diverse evidentiary foundation to map patient profiles.

Clinical adherence was quantified using an all-or-nothing scoring protocol grounded in six foundational principles of tuberculosis treatment: taking the correct dosage without unapproved reduction or omission, maintaining regular daily dosing throughout the entire course, administering medicine once daily on an empty stomach, fulfilling the full prescribed therapeutic duration, attending periodic sputum testing schedules, and arriving promptly for scheduled clinical follow-up appointments. Patients satisfying all six benchmarks were categorized as completely adherent, encompassing 293 individuals or 77.3 percent of the cohort, while those failing any single benchmark were categorized into the non-adherent group, comprising 86 patients or 22.7 percent. This strict binary split reflected pharmacological realities where even isolated deviations increase resistance risks, yielding an inherent baseline class imbalance ratio of approximately 3.41.

To guarantee rigorous validation and prevent data leakage, twelve specific behavioral items directly utilized to score adherence compliance were permanently expunged from the predictor space prior to analytical execution. The dataset was subsequently partitioned using stratified sampling into an 80 percent training partition comprising 303 cases and an independent 20 percent hold-out testing set containing 76 cases. A total of 219 missing data entries located within the training partition were addressed using mode imputation, and these training-derived mode values were then applied directly to handle missing entries within the testing set. Dimensionality reduction was performed strictly inside the training set via the Boruta feature selection framework configured around a Random Forest core, which retained 16 confirmed descriptors covering socio-demographic factors and distinct knowledge attributes.

The sixteen core features selected through Boruta comprised patient age, educational attainment, occupation, economic status, marital status, previous history of tuberculosis diagnosis, overall knowledge of treatment principles, knowledge of proper dosage, knowledge regarding the consequences of missing doses, knowledge of medication timing, knowledge of treatment duration, knowledge of periodic sputum testing schedules, distance to the health center, monthly income, treatment cost affordability, and social support. To address the class distribution skew within the training split, the researchers incorporated the Synthetic Minority Over-sampling Technique, tuning the oversampler to five nearest neighbors with a set random seed. Crucially, synthetic oversampling was confined exclusively to the training split, preventing synthetic data leakage into the independent hold-out test set during final evaluation against an oversampled logistic regression benchmark.

The machine learning architecture was constructed within a Python programming environment utilizing scikit-learn for dataset partitioning and baseline regression, imbalanced-learn for synthetic resampling, xgboost configured with a learning rate of 0.05, a maximum tree depth of 5, and 100 estimators, and boruta_py for feature selection. Performance metrics derived through bootstrapping with 1,000 resamples demonstrated that the oversampled XGBoost model attained an area under the receiver operating characteristic curve of 0.768, a sensitivity of 47.1 percent, and a specificity of 89.8 percent on the independent hold-out partition. Although XGBoost achieved a higher specificity and discriminatory metric than the logistic regression baseline, a paired McNemar test yielded a p-value of 0.4545, indicating no statistically significant difference in absolute classification distribution between the two algorithmic approaches.

Evaluation of model calibration demonstrated that while the tree-boosting framework recorded a lower Brier score of 0.1455 compared to 0.1917 for the linear baseline, the predicted reliability curve departed noticeably from the ideal diagonal trajectory at elevated output probabilities. This miscalibration, manifesting as an overestimation of absolute individual risk, represents an expected mathematical artifact induced by training models on synthetically balanced distributions that artificially distort original event prevalence. Consequently, the authors emphasized that the raw algorithmic probability outputs must be treated as uncalibrated risk indices intended for relative patient stratification rather than direct calculations of true clinical probabilities under natural prevalence rates.

Integrating game-theoretic SHAP interpretability unlocked transparent structural explanations for the boosting ensemble, revealing that global risk predictions were dominated by specific gaps in health literacy rather than purely fixed demographic traits. SHAP beeswarm evaluations highlighted that an absence of patient comprehension regarding periodic sputum testing schedules and fundamental treatment principles was most strongly associated with elevated non-adherence tendencies. Individualized waterfall plots illustrated how localized deficits in testing knowledge and lower formal education compounded log-odds predictions toward high-risk stratification, while multi-variable interaction analyses mapped complex non-linear synergies between advancing age and specific comprehension domains that traditional linear regressions could not readily isolate.

The investigators outlined several key methodological constraints, noting that the cross-sectional design captures observational associations rather than prospective causal determinations. Furthermore, the moderate sample size of 379 patients generated wide confidence intervals for sensitivity, and multicollinearity observed among certain economic indicators could induce credit splitting across tree-based explanation values. The model sensitivity of 47.1 percent indicates that numerous non-adherent cases were missed, potentially reflecting unmeasured transient stressors like sudden financial shocks or emotional crises that cannot be registered via baseline questionnaires. The authors concluded that while human resource demands for counseling remain substantial, identifying specific knowledge gaps via explainable models provides an evidence-based roadmap for tailoring targeted educational interventions in resource-limited clinical environments.

Subject of Research: Explainable machine learning pipeline for tuberculosis treatment adherence factors

Article Title: An explainable machine learning pipeline using XGBoost and SHAP to identify factors associated with tuberculosis treatment adherence in a district health center

Article References: Phuong, H. T., Nguyet, T. T., Viet, N. Q., Thao, T. L. T. N., Ba, T. T. T., Thinh, L. Q., Trang, T. T. T., Phung, N. T., Tam, N. T. M., Huong, P. T. N., Bang, N. T. K., & Loan, T. T. A. (2026). An explainable machine learning pipeline using XGBoost and SHAP to identify factors associated with tuberculosis treatment adherence in a district health center. Discover Public Health, 23(1), Article 1734. https://doi.org/10.1186/s12982-026-03199-w

Image Credits: AI Generated

DOI: 10.1186/s12982-026-03199-w

Keywords: Tuberculosis, Treatment Adherence, Machine Learning, XGBoost, SHAP, Explainable AI, Public Health, SMOTE, explainable, machine, learning, pipeline