Research report: Artificial intelligence-based analysis of digital media engagement and user sentiment using machine learning techniques

The research introduces a framework called the Hybrid Textual Sentiment Engagement Learning Model, or HTSELM, a late-fusion architecture that combines two very different kinds of evidence. The first branch processes text using a TF-IDF representation fed into a linear support vector machine, a workhorse combination for document classification that remains remarkably competitive on moderate-sized datasets. The second branch handles structured metadata—platform, location, language, campaign identifiers, temporal variables and engagement counters—using HistGradientBoosting, a histogram-based gradient boosting method well suited to capturing nonlinear interactions among categorical and numeric features.

Rather than merging these feature families into a single vector space, the model trains each branch separately, converts their outputs into comparable class probabilities, and blends them with a weighted average whose mixing coefficient is tuned on a validation set. When the two branches were given equal weight, performance actually dropped; only when the much stronger text branch dominated did fusion deliver a benefit. On a held-out test set, HTSELM achieved an accuracy of 0.9392, a macro-F1 score of 0.9398, a Matthews correlation coefficient of 0.9043 and a macro ROC-AUC of 0.9923.

Exact five-fold cross-validation produced a mean macro-F1 of 0.9398 with a standard deviation of just 0.0069, indicating that the result was stable rather than a lucky split. The improvement over a text-only support vector machine was small—0.0048 in macro-F1—but consistent across all five cross-validation folds. The most provocative part of the study, however, is not the headline numbers but the audit that preceded them.

Wang examined the Social Media Engagement Dataset, a machine-generated collection of 12,000 records with 28 raw columns, and discovered that the released sentiment_score variable deterministically defines the sentiment_label target: negative labels correspond to scores below roughly −0.20, neutral labels fall in a narrow band around zero, and positive labels begin above roughly +0.20. Similarly, the engagement_rate column agreed perfectly, within rounding tolerance, with a formula computed from likes, shares, comments and impressions. Any model trained with these variables included would not be predicting anything—it would simply be reconstructing the answer key.

By removing sentiment_score, emotion_type, user_past_sentiment_avg and toxicity_score before training the sentiment classifier, and by excluding engagement_rate and all post-response variables from the engagement experiment, the study establishes what its author calls a leakage-controlled baseline—an honest floor rather than an inflated ceiling. The ablation experiments reinforce how much of the predictive signal lives in the words themselves. The explainability analysis tells a consistent story: the most influential TF-IDF terms included phrases such as "okay," "wouldn't recommend" and "highly recommend," recognisable polarity expressions that dominate the classification decision, whereas permutation importance for the structured features was small and dispersed across brand, product, language, campaign and location variables.

In other words, once affective proxies are stripped away, metadata contributes surprisingly little to sentiment discrimination in this corpus. Robustness testing added further nuance. Perturbing numeric features with noise barely dented performance, with macro-F1 values ranging only from 0.9390 to 0.9423 across perturbation levels.

Missing features and text corruption, by contrast, caused gradual but meaningful degradation: at 20 percent perturbation, missing-feature tests dropped macro-F1 to 0.8441 and token-level text noise reduced it to 0.8599. The practical implication is clear—operational reliability of such systems depends far more on the availability and integrity of the text itself than on the precision of the accompanying numbers. The confusion matrix showed the model handled the Neutral class reasonably well, with a recall of 0.9069, and that residual errors were mostly polarity confusions between negative and positive posts.

Perhaps the study's most sobering finding comes from its secondary experiment. When the framework attempted to predict engagement categories using only information available before an audience responds—text, platform, timing and campaign metadata, with all response-derived counters excluded—every model performed close to the three-class chance level of one-third. The best configuration, a structured logistic regression, managed an accuracy of just 0.3479, and HTSELM itself reached only 0.3413 with a Matthews correlation coefficient of 0.0118, effectively zero.

Wang argues this negative result is scientifically valuable: it demonstrates that the pre-response variables in this dataset carry no reliable predictive signal, and it prevents future work from mistaking formula recovery for genuine forecasting ability. The author is careful about the limits of these conclusions. The dataset is synthetic and machine-generated, so its linguistic templates, platform distributions and behavioral patterns may not mirror naturally occurring social media.

The baseline set deliberately excludes transformer models such as BERT and RoBERTa, which could offer richer contextual representations of sarcasm, negation and code-mixing, though the study notes that any such comparison should first be subjected to the same leakage controls. Statistical tests also revealed the constraints of a small evaluation: paired t-tests across five folds favored HTSELM over the text-only baselines, but Wilcoxon and McNemar tests did not reach significance, reflecting limited statistical power rather than contradictory evidence. The study frames explainability not as an afterthought but as part of the experimental protocol, reporting coefficient analyses, permutation importances and error shares alongside performance metrics, in line with human-centered AI principles that demand transparency about data provenance, model limitations and deployment boundaries.

Subject of Research: Machine learning analysis of social media sentiment and engagement with leakage control

Article Title: Artificial intelligence-based analysis of digital media engagement and user sentiment using machine learning techniques

Article References: Wang, X. (2026). Artificial intelligence-based analysis of digital media engagement and user sentiment using machine learning techniques. Discover Artificial Intelligence, 6(1), Article 1448. https://doi.org/10.1007/s44163-026-02430-6

Image Credits: AI Generated

DOI: 10.1007/s44163-026-02430-6

Keywords: artificial intelligence, machine learning, sentiment analysis, social media, target leakage, engagement prediction, late fusion, explainable AI, TF-IDF, gradient boosting, digital media analytics, data leakage