Synthetic benchmark test finds sensitivity-aware graph model improves privacy-leak path ranking

A sensitivity-aware graph neural network called DS-GNN outperformed comparison methods in a synthetic simulation of privacy-leakage tracing across scan-to-order systems. The study evaluated the model on SynOrder-Leak, a generated benchmark containing 50 one-minute snapshots of transactions linking users, merchants, payment platforms and third-party service providers. In five randomized runs, DS-GNN achieved a Precision@10 of 0.72 ± 0.02 and a Mean Reciprocal Rank of 0.61 ± 0.03, according to the research article published in Discover Artificial Intelligence.

The simulated transactions represent QR-code ordering systems in which a transaction may pass through a user terminal, merchant server, payment gateway and one or more analytics or logistics APIs. The researchers framed the threat as “honest-but-curious”: parties follow the protocol, but authorized data transfers can expose more information than was necessary. Under this model, privacy leakage is not attributed to malicious packet injection; it arises when sensitive information reaches an unauthorized destination through otherwise legitimate interactions that resemble routine business traffic.

DS-GNN represents the activity as a time-evolving heterogeneous graph. Each directed edge carries a three-component sensitivity vector for personal identifiers, payment records and location data. Those vectors enter the graph neural network’s attention calculation alongside the representations of the connected nodes, allowing sensitivity to influence message aggregation from the first layer rather than serving only as a final scoring adjustment. The model was trained jointly to identify leakage paths and estimate their risk, with path identification handled through binary cross-entropy and risk prediction through mean squared error.

The benchmark contained 480 nodes—200 users, 120 merchants, 80 payment-platform nodes and 80 third-party service-provider nodes—and 6,240 directed edges marked as access, transfer or storage interactions. It included 156 injected leakage events with ground-truth path labels. Leakage paths ranged from two to eight hops and were generated as structurally valid routes from merchant or gateway nodes to service nodes. The snapshots were divided into 30 for training, 10 for validation and 10 for testing, with leakage events distributed across all three splits. Risk labels combined edge-sensitivity values with interaction-type weights, so path severity was not determined solely by topology or length.

Against four baselines—a rule-based ranker, Path-RNN, a static graph convolutional network and a temporal graph neural network—DS-GNN produced higher path-ranking and risk-estimation scores. Its Recall@10 was 0.65 ± 0.02. The strongest baseline, Temporal-GNN, was exceeded by 0.08 in Precision@10, 0.07 in Recall@10 and 0.08 in Mean Reciprocal Rank, with paired two-sided tests reporting p-values below 0.01. For risk prediction, mean absolute error fell from 0.51 ± 0.03 for Temporal-GNN to 0.43 ± 0.02 for DS-GNN, while root-mean-square error decreased from 0.69 ± 0.03 to 0.58 ± 0.03.

Performance differences were also reported by path length. The static baseline lost roughly half its Precision@10 beyond five hops, while DS-GNN declined by less than five percentage points over the full two-to-eight-hop range. Ablation tests indicated that removing temporal modeling reduced Precision@10 by 0.14, replacing sensitivity-conditioned attention with uniform aggregation reduced it by 0.12, and removing sensitivity encoding reduced it from 0.72 to 0.63. A separate comparison of five ways to incorporate sensitivity found that direct conditioning of the attention coefficient improved Precision@10 by 0.08 over uniform aggregation of sensitivity-tagged messages and by 0.10 over post-hoc rescaling; however, those integration-strategy results came from preliminary single-seed runs and were not presented with the same multi-seed validation as the main comparison.

A counterfactual module re-ran the trained model after removing individual edges or nodes and measured the resulting change in predicted risk. In the simulation, interventions on high-sensitivity payment-processing and third-party data-sharing edges produced larger predicted risk reductions than interventions on auxiliary logging or synchronization edges. Some downstream nodes also had larger modeled effects than structurally central upstream nodes. These are counterfactual contributions within the trained model and synthetic graph, not evidence that changing a particular production connection would necessarily reduce real-world privacy risk.

The findings remain bounded by the study design. SynOrder-Leak was synthetic because the researchers stated that no public benchmark captured the domain’s heterogeneous structure; its sensitivity distributions, leakage injections and one-minute temporal resolution may not represent production logs. The model also assumes that edge-level sensitivity labels are available and accurate, requires all inference-time entity types to have appeared during training, and evaluates edges or nodes individually rather than coordinated sets of interventions. Although cascade effects accounted for less than 5% of the predicted risk change for the ten most important edges in the reported analysis, the authors identify multi-edge intervention analysis and validation on real-world logs as necessary next steps before deployment in compliance workflows.

Subject of Research: Graph neural network evaluation for privacy-leakage traceability in simulated scan-to-order transaction graphs

Article Title: Sensitive-aware graph neural network for privacy leakage traceability in scan-to-order systems

Article References: Zhou, J., & Wu, D. (2026). Sensitive-aware graph neural network for privacy leakage traceability in scan-to-order systems. Discover Artificial Intelligence, 6(1), Article 1276. https://doi.org/10.1007/s44163-026-02198-9

Image Credits: AI Generated

DOI: 10.1007/s44163-026-02198-9

Keywords: DS-GNN, privacy leakage, graph neural networks, scan-to-order systems, synthetic benchmark, path ranking, counterfactual inference, sensitivity encoding, dynamic graphs, risk prediction, Sensitive-aware, graph