MELRec streamlines LLM-based sequential recommendation with end-to-end training

Researchers have introduced MELRec, a modular end-to-end framework that uses large language models for sequential recommendation while simplifying the training process that plagues many existing LLM-based approaches. The work, published in Neural Computing and Applications, addresses two persistent problems in the field: complex architectures and staged optimization, which increase training cost and can introduce representation mismatch between components.

The framework decouples the recommendation pipeline into three distinct layers. The semantic encoding layer initializes item ID tokens with item-title semantics and encodes them using a single GPT-2 model, allowing the language model’s semantic priors to inform how items are represented. The behavior modeling layer then employs lightweight Transformer encoders with additive attention to aggregate a user’s historical interactions. Finally, the interaction prediction layer computes user-item matching scores with a simple dot product.

What distinguishes MELRec from prior LLM-based sequential recommenders is its optimization strategy. Rather than training components separately in stages, the entire system is optimized end-to-end with a single Bayesian Personalized Ranking (BPR) loss. This design allows the semantic encoder, the behavior encoder, and the prediction objective to adapt jointly to the recommendation task, reducing the optimization complexity associated with staged training.

The authors evaluated the framework on four Amazon Reviews datasets, comparing it against traditional sequential models, enhanced sequential models, and recent LLM-based baselines. According to the reported results, MELRec achieved leading or highly competitive performance across these benchmarks.

Beyond accuracy, the experiments indicated that MELRec is robust, stable during training, and efficient. The researchers position the framework as an effective and efficient approach to LLM-based sequential recommendation that lowers the barriers introduced by multi-stage pipelines, offering a simpler path to combining language model semantics with behavioral modeling in recommender systems.

Subject of Research: A modular end-to-end large language model framework for sequential recommendation systems.

Article Title: MELRec: a modular End-to-End framework for large language model based sequential recommendation

Article References: Li, Y., Zhu, R., & Li, L. (2026). MELRec: a modular End-to-End framework for large language model based sequential recommendation. Neural Computing and Applications, 38(20), Article 825. https://doi.org/10.1007/s00521-026-12532-y

Image Credits: AI Generated

DOI: 10.1007/s00521-026-12532-y

Keywords: large language models, sequential recommendation, recommender systems, GPT-2, end-to-end training, BPR loss, Transformer encoders, Amazon Reviews datasets, semantic encoding, MELRec, modular, End-to-End