AI Reads Student Records to Predict English Proficiency in Vocational Colleges

Researchers have built an artificial intelligence framework that predicts how well vocational college students will perform in English, and unlike most prediction models, it explains why it reaches each conclusion. The system, described in the journal Discover Artificial Intelligence, combines the semantic reading ability of a large language model with conventional educational statistics, achieving an accuracy of 92.5 percent and a weighted F1 score of 91.3 percent on a dataset of 200 student records. The work addresses a long-standing gap in educational data mining: models that predict grades well often cannot tell teachers what actually drove the prediction.

The study was motivated by the particular character of vocational education. Students arrive with widely varying language foundations, study habits and levels of classroom engagement. Some perform well on written tests yet struggle orally; others participate actively in class while their grammar and vocabulary lag behind. A single test score therefore cannot capture the general level of English ability, and traditional assessment through exams, oral tests, attendance and project work, while direct, tends to produce a final grade without explaining the reasons behind it. Teachers analyzing many students at once also face a practical time burden that automated tools could ease.

Earlier machine learning approaches to student performance prediction relied on structured features such as historical grades, demographics and attendance. Studies dating back to work on distance learning showed that data-driven models could flag students needing early support, and subsequent reviews catalogued decision trees, neural networks, Bayesian methods and support vector machines as common tools in the field. In English learning specifically, prior studies predicted reading test performance using listening scores and departmental information. But these methods mostly consumed numerical or categorical inputs and could not fully capture the meaning behind a student’s overall learning profile, such as the interplay between oral performance, study time and engagement with learning tools.

The new framework takes a two-track approach. On the first track, each student’s record, containing exam scores, oral assessment scores, class participation, project scores, listening and reading test scores, grammar and vocabulary skills, attendance percentage, learning style, time spent learning and engagement with AI tools, is rewritten as a natural language prompt using a fixed sentence template. A locally deployed Qwen 1.5B model, an open-source causal language model with roughly 1.5 billion parameters, then reads the prompt and produces a semantic embedding. Crucially, the language model’s parameters are frozen; it serves purely as a feature extractor, which avoids costly fine-tuning and suits the small educational datasets typical of vocational colleges.

On the second track, the numerical features are standardized and the categorical learning style, which can be visual, auditory or kinesthetic, is one-hot encoded. Both representations are then projected into a shared 256-dimensional hidden space, where a gated multimodal fusion module blends them. The gate is learned from the concatenation of the two representations and outputs a vector of weights between zero and one: when the semantic profile is more informative, the gate favors the language model branch; when direct indicators dominate, it favors the structured branch. A channel attention module re-weights the fused dimensions, and a residual multi-layer perceptron with three residual blocks and layer normalization classifies each student as beginner, intermediate or advanced.

The experiments drew on a publicly available Kaggle dataset of 200 vocational college English learners. The class distribution was severely imbalanced: 178 students were labeled intermediate, 18 advanced and only 4 beginner. Data was split 70 percent for training, 10 percent for validation and 20 percent for testing, with all preprocessing statistics fitted on the training set alone to prevent information leakage. Training used the AdamW optimizer with a learning rate of 0.001, dropout of 0.25, gradient clipping, and early stopping with a patience of 18 epochs.

The headline numbers look strong, but the authors are careful about what they mean. The proposed model reached 92.5 percent accuracy and a weighted F1 of 91.3 percent, outperforming traditional baselines including logistic regression, gradient-boosted decision trees, k-nearest neighbors, random forests, extra trees and a standard multilayer perceptron. Repeating the full experiment three times with different random seeds yielded a mean accuracy of 0.917 plus or minus 0.014 and a mean weighted F1 of 0.906 plus or minus 0.012, indicating reasonable stability. However, the macro F1 was only 0.569 and balanced accuracy 0.574, because the weighted metrics are dominated by the large intermediate class. The single beginner student in the test set was misclassified as intermediate, and the beginner class achieved an area under the ROC curve of just 0.103 and an average precision of 0.028, confirming that minority-class prediction remains the model’s principal weakness.

The interpretability analysis offers perhaps the most practically interesting findings. The gated fusion weights showed that most average gate values fell below 0.5, meaning the model leaned more on structured features than on language model embeddings, a sensible outcome given that the dataset contains direct indicators closely tied to proficiency. Intermediate students showed a wider spread of gate values, suggesting heterogeneous reliance on the two information sources, while the advanced group displayed a narrower, more stable fusion pattern. Permutation importance ranked reading test scores first, followed by exam scores, with oral assessment, grammar and listening scores contributing positively. Some features, including engagement with AI tools, project scores and class participation, showed near-zero or negative importance, which the authors attribute to noise, redundancy or the dominance of the intermediate class rather than to any causal conclusion about English learning.

Individual case studies illustrate how the framework translates predictions into actionable feedback. One intermediate student with a visual learning style was correctly classified with 0.9887 confidence; the model noted strong project performance, attendance and AI tool engagement against weak reading ability and vocabulary, matching the raw record of an 87 in projects and 88 in attendance but only 54 in reading and 46 in vocabulary, and suggested targeted reading and vocabulary practice. A second intermediate student, strong in listening, participation and attendance but weak in oral assessment at 42 and exam scores at 57, received suggestions to improve speaking and test-taking technique. The authors emphasize that these explanations are supplementary information, not final teaching decisions.

The study’s limitations are stated plainly. With only 200 samples and extreme class imbalance, complex models risk learning dataset-specific patterns, and generalizability to other vocational schools or larger populations has not been validated. The authors note that their contribution lies not in a new language model or fine-tuning algorithm but in an interpretable framework that integrates frozen language model representations with structured educational indicators through adaptive gating, a design choice aimed precisely at small, imbalanced datasets where task-specific adaptation could cause instability or overfitting. Future work, they write, will involve larger and more balanced datasets, additional multimodal information such as learning behavior sequences and text records, and more advanced language models with lightweight adaptation strategies. For now, the results suggest that pairing the semantic breadth of large language models with the transparency of structured educational data can yield predictions that teachers can actually act on, provided the caveats about minority classes and sample size are kept firmly in view.

Subject of Research: Explainable large language model framework for predicting English proficiency of vocational college students

Article Title: Predicting English learning performance in vocational colleges using explainable large language models

Article References: Ju, L. (2026). Predicting English learning performance in vocational colleges using explainable large language models. Discover Artificial Intelligence, 6(1), Article 1595. https://doi.org/10.1007/s44163-026-01934-5

Image Credits: AI Generated

DOI: 10.1007/s44163-026-01934-5

Keywords: large language models, English proficiency prediction, vocational education, educational data mining, explainable AI, multimodal fusion, student performance, machine learning, class imbalance, interpretability, Qwen, personalized learning