Hybrid LLM and SHAP framework explains cyberbullying predictions

A study of 5,000 labeled Turkish tweets found that combining machine-learning classifiers with two forms of explainable artificial intelligence produced complementary views of cyberbullying predictions. Researchers compared seven conventional classifiers and found that gradient boosting achieved the highest mean weighted accuracy across five cross-validation folds, at 85%, followed by a decision tree at 82.8%. In a separate evaluation with 100 participants, SHAP feature attributions were judged more detailed and informative, while natural-language explanations generated by a locally running large language model were viewed as clearer and more user-oriented.

The findings suggest that technical attribution and plain-language rationales can serve different purposes, although the evidence comes from one dataset and a simulated platform rather than a live social-media service. The project was designed to address two related problems: identifying cyberbullying in social-media messages and making a classifier’s decisions understandable. The authors note that harmful communication is not reducible to offensive words alone.

Repeated harassment, engagement patterns, user relationships and a sender’s position within a social network can provide relevant context. Linguistic and stylistic variables were extracted from the text, including word length, emoji use, punctuation patterns and the frequency of slang. This feature set allowed the models to consider both the content of a message and the social behavior surrounding it.

Exploratory analysis identified several associations within the dataset. Messages from accounts with more followers, longer histories or greater posting activity were more often categorized as non-cyberbullying, while messages containing multiple slang terms were more strongly associated with the cyberbullying class. Accounts using a high frequency of slang and short words appeared particularly often among messages linked to suspected fake or aggressive accounts.

These findings are observational patterns within the research data, not proof that an individual feature causes cyberbullying or that established accounts behave more responsibly in general. The researchers also cautioned that correlated variables can complicate SHAP attribution because importance may be distributed among related features, causing an individual variable to appear less influential than it is within the larger group. For classification, the team implemented a decision tree, random forest, logistic regression, k-nearest neighbors, gradient boosting, linear support vector classification and a multilayer perceptron.

The models used the cleaned combination of social, behavioral and linguistic variables, with hyperparameters tuned through grid search inside a five-fold cross-validation procedure. Evaluation included weighted accuracy, precision, recall, F1-score, true-positive rate, false-positive rate, training time and classification time. In addition to gradient boosting, the decision tree and random forest performed moderately well in this evaluation.

The k-nearest neighbors, linear support vector classification and multilayer perceptron had lower weighted accuracies, while the multilayer perceptron required more training time. A Detoxify transformer classifier, included as a text-focused comparison, also trailed some models using social and user features, indicating that those features contained predictive information not captured by message content alone. The framework’s explainability module operated after classification through two complementary layers.

A locally running LLM received the evaluated message, the predicted class and the relevant feature values, then generated a natural-language justification. A constrained system prompt required the model to rely only on that supplied information, which the authors described as a safeguard against introducing unsupported material. The reference implementation used LLaMA 3.2 3B through Ollama, allowing explanations to be generated in a controlled local environment rather than sending message data to an external service.

SHAP, the model-agnostic Shapley additive explanations method, separately estimated the contribution of individual features to a prediction. The system displayed those contributions with beeswarm, force and waterfall plots, providing an aggregate view, a local comparison with a baseline and a step-by-step decomposition of an individual result. In a demonstration built as a simulated social platform, an offensive reply was classified as cyberbullying and hidden behind a warning.

Its generated explanation highlighted signals including slang, short words, the absence of emojis and limited account engagement. The associated SHAP plots showed how positive contributions from some features moved the output toward the cyberbullying class, while negative contributions from others moved it toward the non-cyberbullying class. These examples illustrate the intended division of labor: the LLM converted model inputs and output into text that could appear beneath a message, while SHAP exposed the numerical contribution of individual variables for analysts seeking a more detailed account.

The demonstration was a controlled testbed, not a deployment on an operating social platform with real users. To compare the explanations, the researchers used 12 properties covering content, presentation and the user perspective. These included correctness, relevance, completeness, clarity, coherence, confidence, trust and support for decision-making.

The survey involved bachelor’s and master’s students in computer and management engineering, information-technology professionals and university researchers. Based on self-reported AI knowledge and social-media use, the researchers divided the 100 respondents into 68 “basic” users and 32 “professional” users. Messages from the test set were used to generate and compare explanations, not to re-evaluate the classifiers’ predictive accuracy.

Participants rated the SHAP and LLM outputs against the same classification results using five-point scales. Both participant groups regarded explanations as important for transparent moderation, but their preferences differed with technical expertise. Basic users rated SHAP as more informative on several content-related properties, although the gap from the LLM method was relatively narrow, and they considered the two approaches similarly user-oriented.

They gave SHAP higher confidence ratings because its numerical indicators appeared more reliable. Professional users showed a clearer preference for SHAP’s completeness, consistency and feature-level precision. At the same time, both groups favored LLM explanations for presentation and structure, reflecting the accessibility of natural-language output.

Correctness and coherence received consistently positive scores for the LLM explanations, with no evident signs of divergence from expected model behavior in the study’s evaluation. Overall, the ratings supported pairing quantitative attribution with a more accessible textual account rather than treating one explanation method as sufficient for every audience. The study’s conclusions are promising but bounded.

The analysis used one dataset of 5,000 posts in one language, and the authors identify possible dataset bias as a threat to generalizability. Gradient boosting was the strongest reference classifier, but the results do not establish that the same model will perform equally well on other platforms, languages, evolving slang or adversarial behavior. The 100-person survey included participants with different levels of expertise, but its ratings measure perceived explanation quality rather than independently demonstrate that either method improves real-world moderation decisions.

The authors also identify unresolved issues involving fairness, disparate impacts, uncertainty and continuous model monitoring. Future work is planned on adaptive explanations, user feedback, additional language models, uncertainty-aware prompting and hierarchical grouping of correlated features. The clearest demonstrated result is therefore a technically workable pairing: social and linguistic features can support cyberbullying classification, SHAP can expose granular evidence for technical analysis, and constrained LLM outputs can translate model evidence into explanations intended for broader audiences.

Subject of Research: Explainable machine-learning methods for detecting cyberbullying in social-media posts

Article Title: A hybrid framework combining large language models and Shapley-based methods for explainable cyberbullying detection

Article References: Loseto, G., Gramegna, F., Pinto, A., Scioscia, F., & Ruta, M. (2026). A hybrid framework combining large language models and Shapley-based methods for explainable cyberbullying detection. Discover Artificial Intelligence, 6(1), Article 1158. https://doi.org/10.1007/s44163-026-02204-0

Image Credits: AI Generated

DOI: 10.1007/s44163-026-02204-0

Keywords: cyberbullying detection, explainable AI, large language models, SHAP, social media moderation, machine learning, feature attribution, human-centered evaluation, hybrid, framework, combining, large