Multimodal model links tourist images, reviews and ratings to satisfaction

A multimodal machine-learning system that combines tourist photographs, written reviews and numerical satisfaction ratings achieved 97.68% accuracy when classifying polarized visitor sentiment in a balanced dataset drawn from a public Kaggle tourism collection. The model reported in the study used BERT to encode review sentiment, ResNet-50 to extract visual features and an optimized categorical-boosting classifier to predict whether a tourist’s rating indicated low or great satisfaction. Comfort and visual appeal emerged as the strongest model-associated indicators, although the researchers cautioned that feature importance represents predictive relationships rather than proof that either factor causes satisfaction.

The work addresses the incompleteness of analyzing tourism through only one type of evidence. A text review may describe comfort but say little about scenery, while an image can communicate visual appeal without revealing how a visitor felt about congestion, service or cultural atmosphere. Ratings provide a standardized measure of satisfaction but do not themselves explain which parts of an experience shaped the score. Each record in the Multimodal Tourism Experience Dataset combined an image, a user-generated prose review and a satisfaction score from the same tourist, with the broader collection containing 18,300 examples from metropolitan, natural and cultural tourism settings.

For the experiment, the researchers filtered the collection to 2,000 observations containing all three required forms of information and used stratified sampling to preserve class balance. Ratings of 1 or 2 were labeled low satisfaction, ratings of 4 or 5 great satisfaction, and rating 3 was treated as neutral and excluded from the binary task. The paper describes both an 80% training and 20% testing division with cross-validation for tuning and a separate 70% training, 15% validation and 15% testing arrangement in its experimental setup. A fixed random seed of 42 was used, and preprocessing transformations were fitted only on training data so that information from the test set did not influence feature engineering or model construction.

Images were denoised with Gaussian filtering, resized to 224 by 224 pixels and normalized using ImageNet means and standard deviations. The pretrained ResNet-50 convolutional neural network then extracted a fixed-length, 2,048-dimensional representation from its final global-average-pooling layer. These visual features were intended to encode properties such as scene composition, environmental context and aesthetic appearance. For text, the researchers fine-tuned the 12-layer, 768-hidden-unit BERT-base-uncased model with sequences capped at 128 tokens, using the embedding associated with the classification token as a contextual representation of each review. Numerical ratings were transformed with min-max scaling, expressed in the paper as the value minus its minimum divided by the difference between its maximum and minimum.

The three feature types were mapped into a shared latent space and combined through early feature-level concatenation. In this design, projected text, visual and rating embeddings form one vector before the classifier begins learning, allowing relationships among the modalities to be modeled jointly rather than combining separate predictions at the end. The researchers selected this approach because it can expose cross-modal interactions during training at lower computational cost than late or hybrid fusion. The resulting representation was passed to Efficient Categorical Boosting, a gradient-boosting framework that builds trees sequentially and handles categorical variables through specialized encoding.

Model tuning was handled by Dynamic Differential Annealing, or DDA, a hybrid of Differential Evolution and Simulated Annealing. Differential Evolution generated candidate hyperparameter settings through mutation and crossover, providing broad exploration of the search space. Simulated Annealing then refined promising candidates and could probabilistically retain an inferior configuration early in the search, reducing the chance of settling immediately on a poor local optimum. The optimizer considered the learning rate, tree depth, boosting iterations, L2 regularization, random strength, bagging temperature, border count, feature-sampling ratio and leaf-estimation settings. Candidate configurations were evaluated using validation accuracy, after which the selected parameter set was used to train the final classifier and assess performance on the independent test set.

On the test set, the proposed DDA-ECatBoost model recorded an accuracy of 0.9768, precision of 0.9647, recall of 0.9555 and F1 score of 0.9651. In the study’s comparison with neural-network alternatives, a CNN achieved 0.7906 accuracy, while a BERT-BiLSTM-CNN-Attention model reached 0.9313. The proposed system also produced a reported average precision of 0.98, and bootstrap resampling of the held-out test set yielded narrow 95% confidence intervals for the principal metrics. Training and validation accuracy rose above 0.90, while both loss curves generally declined, which the authors interpreted as evidence of convergence without severe overfitting.

Model interpretation focused on the features most useful for prediction. Comfort-related textual signals and visual appeal ranked among the most influential variables, and the feature-importance visualization indicated that sentiment features carried both positive and negative predictive associations. A correlation analysis also showed strong dependencies between visual beauty and comfort, while a two-dimensional t-distributed Stochastic Neighbor Embedding plot separated most satisfied and unsatisfied observations after fusion. Error analysis indicated that conflicting cues accounted for many mistakes: visually attractive destinations could accompany neutral or negative reviews, while short positive comments could coexist with lower numerical ratings. These are model-level associations and prediction errors; the study did not establish that changing comfort or scenery would directly cause ratings to improve.

The results should be interpreted in light of the study’s design and data limitations. All experiments relied on one publicly available Kaggle dataset, which may be region-specific and may not represent the full diversity of destinations, cultures, languages and visitor populations. User-generated content can introduce demographic and platform biases, and real-world records may contain noise or missing modalities that the filtered benchmark does not fully capture. Although the researchers performed threshold-sensitivity tests and reported stable performance under alternative definitions of the positive class, those tests cannot replace external validation. The computational demands of BERT and ResNet feature extraction, along with the framework’s scalability to larger and more complex datasets, remain practical constraints.

Overall, the study presents multimodal early fusion as a promising way to evaluate recorded tourism satisfaction while retaining interpretable boosting rather than relying only on opaque deep networks. The strongest result is not a causal conclusion about what makes tourists happy, but a modeling observation: when aligned reviews, images and ratings were available, combining them produced high performance on the selected benchmark, with comfort and visual appeal carrying substantial predictive weight. Cross-regional testing, uncertainty-aware handling of contradictory modalities, attention-based fusion, and the incorporation of additional sources such as video, audio or geospatial information were identified as priorities for determining whether the approach can operate reliably beyond the dataset studied.

Subject of Research: Multimodal machine learning for predicting tourist satisfaction from images, reviews and ratings

Article Title: Multimodal machine learning-supported comprehensive evaluation of tourism experience

Article References: Jia, Y. (2026). Multimodal machine learning-supported comprehensive evaluation of tourism experience. Discover Artificial Intelligence, 6(1), Article 1056. https://doi.org/10.1007/s44163-026-02117-y

Image Credits: AI Generated

DOI: 10.1007/s44163-026-02117-y

Keywords: tourism analytics, multimodal machine learning, tourist satisfaction, BERT, ResNet-50, categorical boosting, sentiment analysis, visual features, user-generated content, Multimodal, machine, learning-supported