Static Feature Models Outperform Deep Sequences in Retail Lifetime Value Forecasts

A rigorous evaluation of customer lifetime value modeling has revealed that domain-engineered static features outperform complex deep temporal neural networks across grocery retail environments when customer cohorts remain modest. Publishing in Frontiers in Artificial Intelligence, an international research team led by Denis Kovalenko and Sheikh Md Faysal analyzed 2,581,266 grocery transactions from 2,500 households to assess whether temporal deep learning surpasses traditional summary metrics. The researchers introduced the Cross-Channel Temporal Fusion Network, a dual-branch architecture combining weekly sequence tensors with static customer aggregates through an adaptive sigmoid gate. A four-way ablation established that an engineered static multi-layer perceptron achieved a Spearman rank correlation of 0.8060 and root mean squared error of 485.60, outperforming the full gated sequence model at 0.7998. Concurrently, the investigation uncovered a striking asymmetry: marketing channel exposures accounted for only 1.50 percent of predictive permutation importance, yet rigorous econometric evaluation demonstrated that omnichannel marketing engagement generated a causal lift of 12 to 23 percent in customer lifetime value.

Traditional customer lifetime value modeling condenses longitudinal purchasing records into static recency, frequency, and monetary summaries, as seen in classical Pareto/NBD and BG/NBD models. While computationally practical, this aggregation is inherently lossy because it discards granular purchasing trajectories, promotional reaction lags, and spending bursts that signal enduring loyalty. In principle, temporal deep sequence architectures should recover this lost structure and provide enhanced predictive accuracy. However, retail sequences present severe domain challenges that complicate recurrent neural learning. Weekly grocery purchasing matrices are heavily zero-inflated, with 66.86 percent of the 78-week calibration tensor containing no transactions. In addition, shopping visit densities vary across three orders of magnitude, ranging from a single trip to 1,298 trips per household. Crucially, because unique customers represent the true independent units rather than raw transaction records, the effective sample size is limited to 2,500 households, placing recurrent models at their lower stability bound for pattern generalization.

To address these data characteristics, the investigators developed the Cross-Channel Temporal Fusion Network using a parallel dual-branch architecture. The temporal branch processes an 11-channel sequence tensor across a 78-week window, beginning with a one-dimensional convolutional layer using a 5-week kernel and Gaussian Error Linear Unit activations to capture monthly purchasing cycles. This representation passes into a two-layer bidirectional gated recurrent unit with a hidden size of 64, followed by four-head attention pooling that generates a unified temporal vector and interpretable weekly attention weights. Simultaneously, the static branch processes 74 engineered aggregate features through an inverted-bottleneck residual multi-layer perceptron expanding from 74 to 128 before projecting to 64 dimensions with an active skip connection. An adaptive per-household sigmoid gate dynamically blends the two 64-dimensional vectors based on household sequence information. The merged representation feeds into a shared trunk network with multi-task heads predicting customer retention through cross-entropy loss and conditional spend via masked Smooth-L1 loss.

The empirical validation leveraged the complete Dunnhumby Complete Journey panel, containing 2,581,266 transaction records spanning 711 consecutive days across 92,015 products and 582 stores. The researchers implemented a disjoint timeline design, utilizing days 1 through 547 as a 78-week calibration period to construct all sequence tensors and static predictors, while evaluating lifetime value over an independent 164-day holdout window from days 548 through 711. To augment the sequence observations without creating artificial customers, the team applied feature engineering across four distinct calibration cut-points at days 428, 468, 508, and 547, yielding 10,000 temporal observations across the 2,500 unique households. Model validation employed GroupKFold cross-validation with five folds partitioned strictly by household key, ensuring no consumer appeared across both training and validation splits. Furthermore, all standardization, median imputation of missing demographics, and categorical encodings were derived exclusively within training folds, establishing rigorous safeguards against information leakage from holdout periods.

To isolate the performance of each neural component, the authors conducted a four-way ablation under standardized training hyperparameters using the AdamW optimizer. Configuration A, relying strictly on the 74 static features and residual multi-layer perceptron, demonstrated the strongest predictive performance among individual neural models, achieving a root mean squared error of 485.60 and a Spearman rank correlation of 0.8060. Configuration B, which isolated the temporal convolutional and recurrent encoder, suffered marked degradation, recording an error of 554.47 and a correlation of 0.7891. Configuration C, combining both branches via fixed-ratio concatenation, reached an error of 498.58 and a correlation of 0.7956. Configuration D, the complete gated CCTFN model, achieved an error of 498.54 and a correlation of 0.7998. The ablation clearly answered the primary research question: at an effective sample size of 2,500 unique households, weekly sequence tensors do not provide incremental predictive accuracy beyond domain-engineered static summaries. The hand-crafted RFM metrics and trend variables already capture the necessary trajectory signals.

Although the gated temporal model did not surpass the purely static network, the empirical comparison between Configuration D and Configuration C highlighted the specific mechanical value of learned gating. Gated fusion outperformed naive fixed-weight concatenation by a statistically significant margin of 0.0042 Spearman rank points, with a 95 percent confidence interval from 0.0028 to 0.0056 and a paired t-statistic of 8.3 across folds. Rather than magnifying signal for high-frequency consumers, the gate served primarily as a mechanism for graceful degradation, preventing sparse and uninformative sequence representations from actively degrading final forecasts. Empirical gate values ranged from 0.12 to 0.71 with a median of 0.41, showing a statistically significant negative correlation of negative 0.44 against household sequence sparsity. For the densest shoppers, the gate assigned an average temporal weight of 0.52, whereas for the sparsest quartile of consumers exhibiting over 82 percent inactive weeks, the gate suppressed temporal inputs down to 0.29.

Beyond the four ablation variants, the authors evaluated fifteen distinct predictive algorithms spanning tree ensembles, classical stochastic benchmarks, and linear models across identical cross-validation splits. Well-tuned tree ensembles dominated the predictive standings: Random Forest achieved the lowest overall root mean squared error of 483.32 with a Spearman correlation of 0.8370, while ExtraTrees recorded a correlation of 0.8413 and Hurdle gradient boosting reached 0.8321. Classical parametric BG/NBD models combined with Gamma-Gamma spend layers yielded a correlation of 0.7933 and an error of 532.50. To capture potential model complementarity, the researchers constructed a stacked ensemble utilizing a non-negative Ridge meta-learner trained on out-of-fold predictions. The stacked model attained the highest overall performance across the entire benchmark, reaching a Spearman correlation of 0.8414, a Gini coefficient of 0.8754, and an error of 495.43. In this stacked ensemble, Random Forest received the largest weight at 0.4740, followed by Ridge regression at 0.3627, Hurdle gradient boosting at 0.1127, and CCTFN at 0.0706, demonstrating that the gated sequence model contributed non-redundant diversification value.

To determine the specific features driving predictive power, the researchers calculated model-agnostic permutation feature importance over eight random permutations using a LightGBM architecture on validation data. The findings revealed that predictive performance was heavily dominated by recent monetary momentum. The static feature capturing spending over the final 90 days of calibration registered an importance score of 0.2855, ranking more than 16 times higher than the second-ranked predictor, demographic age band, which scored 0.0722. Calibration frequency, total calibration spending, recency, and basket size formed the next tier of influential variables. Conversely, all seven marketing channel-exposure variables combined—including direct mail campaigns, coupon redemptions, redemption rates, promotional mailers, in-store displays, loyalty program participation, and composite channel breadth—accounted for only 1.50 percent of total permutation importance. Cross-fold sensitivity analyses across folds two and five confirmed the robustness of this small contribution, yielding shares of 1.52 percent and 1.47 percent.

The marginal predictive role of channel variables prompted the researchers to perform econometric causal validation, examining whether marketing contact drove incremental spending or merely reflected selective targeting. Utilizing propensity score matching across eleven pre-treatment covariates, the team evaluated the average treatment effect of omnichannel engagement, defined as interaction across three or more channels. The propensity model achieved an area under the curve of 0.8998, confirming non-random campaign assignment, and matched 1,045 of 1,047 households within a 0.5073 caliper. Post-matching covariate balance was exceptional, with maximum standardized mean differences falling from 0.42 to 0.06. While an unadjusted naive comparison indicated a difference of 136.98 dollars, propensity score matching revealed an average treatment effect on the treated of 193.55 dollars, reflecting a 22.8 percent uplift over the mean lifetime value of 848.98 dollars. This discrepancy identified a negative selection bias of negative 56.57 dollars, demonstrating that the retailer had targeted households with lower baseline spending. Difference-in-differences analysis corroborated this causal impact at 100.30 dollars following formal verification of parallel pre-treatment trends with an event-study F-test p-value of 0.91.

These findings establish critical empirical guidelines for retail data science and resolve a persistent tension between predictive modeling and causal intervention. In retail sequence applications with high zero-inflation and sample sizes near 2,500 households, feature-engineered static aggregates provide superior ranking performance compared to recurrent neural networks, which authors project will require sample scales of 20,000 customers before temporal representations surpass static baselines. Concurrently, the sharp contrast between a 1.50 percent predictive importance share and a 12 to 23 percent causal revenue uplift demonstrates that predictive ranking must not be confused with intervention value. Purely predictive models systematically undervalue marketing channels because spending momentum dominates cross-sectional ranking, yet those same channels serve as effective causal drivers for improving customer value. While study limitations include 66.86 percent sequence sparsity, absent digital tracking channels, and mild temporal dependence from multi-origin slicing, the research provides a vital diagnostic boundary for modern enterprise customer analytics.

Subject of Research: Comparative predictive and causal evaluation of temporal deep learning for retail customer lifetime value

Article Title: Cross-channel temporal fusion for retail customer lifetime value prediction: architecture, ablation, and causal validation of a gated BiGRU fusion network

Article References: Faysal, S. M., Sultana, S., Haldar, U., Miah, M. R., Akther, A., Hossain, M. R., Hossain, M. M., Acharjee, S., & Kovalenko, D. (2026). Cross-channel temporal fusion for retail customer lifetime value prediction: architecture, ablation, and causal validation of a gated BiGRU fusion network. Frontiers in Artificial Intelligence, 9, Article 1956496. https://doi.org/10.3389/frai.2026.1956496

Image Credits: AI Generated

DOI: 10.3389/frai.2026.1956496

Keywords: customer lifetime value, temporal fusion network, bidirectional GRU, attention pooling, Dunnhumby Complete Journey, propensity score matching, difference-in-differences, ablation study, omnichannel marketing, permutation feature importance, sigmoid gating, GroupKFold cross-validation, spend momentum, negative selection bias, stacked ensemble, non-contractual retail, multi-task learning