BERT-RF model links hotel review sentiment to satisfaction drivers

A sentiment-analysis system that combines contextual language representations with statistical features achieved 90.1% accuracy in classifying hotel reviews, according to a study of 42,815 comments from 32 hotels on China’s Ctrip, Meituan and Fliggy platforms. The researchers also used partial least-squares structural equation modeling, or PLS-SEM, to connect review-derived measures with customer satisfaction. In the model, perceived value had the strongest association with satisfaction, while unresolved complaints showed a negative direct relationship with loyalty. The work presents a potential way to turn online feedback into operational signals for hotels, although its results are based on Chinese hotel listings collected from January 2023 through December 2024 and do not establish that the proposed management interventions will cause better business outcomes.

The team began by building a dataset from web-crawled reviews containing booking time, room type, written comments and ratings. Its cleaning pipeline removed duplicate submissions by the same user within 24 hours, non-Chinese and non-textual entries, suspected promotional reviews and platform-flagged spam. Because a numerical rating sometimes conflicts with the sentiment expressed in a comment, the researchers treated the ratings as provisional labels. Reviews with strongly opposing rating-based and lexicon-based signals were reviewed independently by two experienced hotel managers while the rating was hidden; unresolved cases were excluded. This arbitration changed the labels of 4.3% of the raw samples and removed 1.2%, producing inter-annotator agreement measured by Cohen’s kappa of 0.87.

After annotation, roughly 68% of the reviews were positive, 20% neutral and 12% negative. To prevent the classifier from favoring the majority class, the researchers used stratified downsampling to create a balanced training dataset of about 36,000 reviews, with approximately 12,000 in each sentiment category. The data were divided 7:2:1 into training, validation and test sets, and the test set contained about 3,600 balanced examples. This design makes the reported overall accuracy more informative than performance on a naturally skewed distribution, but it also means the figures do not describe how the system would perform on a live stream in which most incoming reviews are positive.

At the center of the analysis is BERT-RF, a hybrid architecture in which a pre-trained Chinese BERT model supplies contextual embeddings and a random forest combines those embeddings with complementary lexical and statistical signals. Each review was tokenized and truncated or padded to 128 tokens. The pooled output from BERT’s classification token formed a 768-dimensional semantic vector. In parallel, the researchers constructed a 106-dimensional statistical vector: a 100-dimensional average of pretrained Word2Vec word vectors, the mean of the five highest TF-IDF values and one domain-lexicon sentiment score. Concatenation produced an 874-dimensional input to a random forest with 200 trees and Gini impurity used for splitting.

The model was trained using a feature-importance-guided fusion strategy rather than treating every input as equally informative. The random forest’s learned importance statistics were used to emphasize features that added discriminative value while reducing the influence of redundant or weak signals. The authors also froze BERT during random-forest training, so no gradient updates were passed from the forest into the language model. The authors say this separation was intended to limit overfitting on a dataset of short reviews and to prevent dominant contextual representations from suppressing domain-specific cues. In practice, a base BERT configuration reached 88.0% accuracy, while adding the statistical features raised overall accuracy to 90.1%; a McNemar test reported a chi-square value of 14.73 and p below 0.001 for the difference.

Feature-combination tests showed why the hybrid model may be useful for hotel language. BERT alone produced approximately 84% accuracy in one comparison, rising to about 86% with a domain-emotion lexicon, 85% with TF-IDF, 88% with Word2Vec and about 90% when all features were included. An ablation analysis further reported that the dynamic-weighting model outperformed naive concatenation of the same feature set by 1.4 percentage points in accuracy and 0.017 in macro F1. Removing the lexicon score reduced the F1 score for negative-review identification from 0.862 to 0.847, suggesting that domain wording can complement contextual encoding. The complete model reported 93% recall for positive reviews, 91% F1 overall and 82% accuracy for neutral reviews, but the source does not report performance broken down by hotel, platform or room type.

For the second part of the study, the researchers adapted the Chinese Customer Satisfaction Index model to hotels, defining seven latent constructs: brand image, customer expectations, quality perception, value perception, customer satisfaction, customer complaints and customer loyalty. The model tested 13 hypothesized paths, with 12 found statistically significant at p below 0.05 after 5,000 bootstrap resamples. The sole non-significant path was from brand image to loyalty, with a coefficient of 0.068 and p of 0.073. Reliability and validity checks were generally strong: Cronbach’s alpha and composite reliability exceeded 0.70, average variance extracted exceeded 0.50, and the highest heterotrait-monotrait ratio was 0.724. The study did not experimentally intervene on hotel management, so these results describe relationships within the specified model rather than causal effects.

Value perception, defined as a customer’s subjective assessment of cost-effectiveness, had the largest reported correlation with satisfaction, at 0.774. Satisfaction correlated with loyalty at 0.749, and value perception correlated with loyalty at 0.702, consistent with a modeled “cost-effectiveness to satisfaction to loyalty” transmission pattern. Quality perception had a positive relationship with value perception of 0.342, while the correlation between expectations and quality perception was 0.303. Complaints correlated negatively with value perception at −0.373 and with satisfaction at −0.352. Although complaints showed a positive raw correlation with loyalty of 0.327, the structural model produced a negative direct coefficient of −0.242 after accounting for satisfaction and value perception. The authors caution that this bivariate result may reflect the greater tendency of active OTA users both to revisit hotels and to report failures, rather than any benefit produced by complaining.

The hotel-specific scores identified several shortfalls in the sampled properties. Customers rated expected hardware facilities at 7.832, but perceived value at 6.325, indicating a sizable expectation–experience gap. Room cleanliness scored 6.790, whereas bathroom experience and front-desk response were lower, at 6.123 and 6.132 respectively. These are scores for the study sample, not a ranking of hotels or universal service benchmarks. Based on the model, the authors suggest using sentiment analysis to flag high-intensity negative reviews, grouping recurring low-scoring themes for service planning, examining pricing and added benefits where value perception is weak, and monitoring complaints as early warning signals. These are operational proposals derived from the observed relationships, not outcomes demonstrated by a controlled trial. Cultural and regional moderators were not included, and the authors say future work should use time-series methods to track how satisfaction drivers evolve.

Subject of Research: Hybrid sentiment analysis and satisfaction modeling for hotel online reviews

Article Title: Emotion analysis and management improvement of hotel user online reviews based on BERT-RF and PLS-SEM integration

Article References: Xia, X., Qu, C., & Qu, Y. (2026). Emotion analysis and management improvement of hotel user online reviews based on BERT-RF and PLS-SEM integration. Discover Artificial Intelligence, 6(1), Article 1174. https://doi.org/10.1007/s44163-026-02199-8

Image Credits: AI Generated

DOI: 10.1007/s44163-026-02199-8

Keywords: hotel reviews, sentiment analysis, BERT, random forest, PLS-SEM, customer satisfaction, online feedback, hospitality management, structural equation modeling, Emotion, analysis, management