New AI Model Scales Up Learning for Knowledge Graphs Full of Unseen Entities

Researchers report a new framework called VMCL that improves how machine learning systems handle knowledge graphs when the entities they must reason about were never seen during training. The work, published in the journal Data Science and Engineering, tackles a problem known as inductive knowledge graph embedding, where a model must learn structural patterns from one graph, the source, and apply them to a second graph, the target, whose entities are entirely different.

Knowledge graphs organize facts as triples of a head entity, a relation and a tail entity, such as a company linked by a headquartered-in relation to a city. Most embedding methods operate in a transductive setting, assuming that every entity to be tested was already seen during training, so learned representations can simply be reused. But real knowledge graphs evolve, and new graphs emerge populated by novel entities. In the inductive setting, the representations of old entities are useless for the new ones. What must transfer instead are structural patterns: if an entity in the target graph has the same kinds of relations as, say, Facebook in the source graph, the model should infer it may also be a company.

The authors identify three persistent weaknesses in prior approaches. First is sparsity: in the four standard inductive benchmarks N1 through N4, the degree of 65, 47, 46 and 42 percent of entities respectively is below three, meaning most entities have very few connections from which to learn. Second is implicit transfer: methods such as MorsE, which uses meta-learning to learn transferable, entity-independent meta knowledge, move patterns between source and target graphs implicitly, without any explicit joint comparison across entities and graphs. Third, prior studies have largely overlooked scaling behavior, leaving it unclear how model capacity shapes inductive transfer.

VMCL addresses the first two problems with two coupled components. A representation generation module uses a graph-guided variational autoencoder, built on graph neural networks, to produce generated representations of entities alongside their encoded ones. These generated variants are similar to the originals but not identical, and they augment the representation space with complementary features, directly countering the thin evidence available for sparse entities. The encoder learns a latent Gaussian distribution for each entity, and the decoder samples from that distribution through the reparametrization trick to reconstruct generated representations, trained with a combined reconstruction and KL-divergence loss.

On top of that generative layer, the framework introduces two contrastive learning objectives that make the transfer mode explicit. One objective works across entities inside a meta-knowledge graph, pulling true triples closer while pushing apart corrupted versions in which the head entity, tail entity or relation has been replaced by a negative sample. The other works across meta-KGs, linking a meta-KG with its generated counterpart through an added relation and contrasting an entity against negative entities drawn from other meta-KGs. Meta-KGs themselves are subgraphs randomly sampled from the source graph with entity identifiers re-labeled, following the MorsE setup, so the model learns patterns rather than memorizing particular entities. A gradient analysis in the paper shows how these losses implement a pull toward positive samples and a push away from harder negatives, transferring features in both directions.

To probe scaling, the team extended the framework so that each entity can receive multiple generated representations, fused by three strategies: ADD, which sums representations element-wise; CAT, which concatenates them and projects back to the original dimension; and IND, which trains each generated representation independently. The authors then systematically scaled three capacity knobs: the embedding dimension, varied over 16, 32, 64 and 128; the number of meta-KGs, varied from 100 up to 100,000; and the number of generated representations, varied from 1 to 10. Performance was evaluated from two perspectives, outside the data coverage, where only pretraining on the source graph is used and test entities are unseen, and inside the data coverage, which adds fine-tuning on the target graph.

The scaling results show smooth, predictable gains rather than sudden emergent jumps. As the number of meta-KGs increases, the H@1 score outside the data coverage rises sharply at first and then stabilizes; on the F1 dataset it climbs rapidly from 5 percent to 60 percent before slowly reaching 70 percent, while the inside score increases slowly and steadily. Increasing the dimension size produces an approximately linear upward trend in H@1 across both perspectives for most datasets. Scaling generated representations from 1 to 3 yields a pronounced improvement, but from 3 to 10 the curve flattens, which the authors interpret as a transition from an effective scaling regime to a weak-scaling regime where redundancy limits further gains. They explicitly position these findings as standard scaling behavior, consistent with smooth power-law trends, rather than phase-transition-like emergent phenomena.

The headline accuracy results come from experiments on twelve benchmark inductive knowledge graphs, W1-W4, F1-F4 and N1-N4, evaluated by mean reciprocal rank and hits-at-N on link prediction in the target graph’s test set, with each test triple ranked among 50 randomly sampled negatives five times. The improved VMCL with multiple generations and the CAT fusion strategy beats the state-of-the-art MorsE baseline by clear margins in H@1: gains of 6.0, 2.4, 1.8 and 0.9 points on F1-F4, 6.4, 6.7, 1.8 and 3.5 points on N1-N4, and 9.2, 1.3, 0.8 and 1.9 points on W1-W4. The improvements are largest on sparse graphs such as F1-F3 and N1-N4 and smaller on the denser F4, which the authors say matches their motivation of alleviating sparsity. Ablations reinforce the design: removing the representation generation module costs 10.56 to 12.21 H@1 points on F1-F4, and the contrastive objectives matter most on the sparsest datasets, where extra negative samples counteract the missing-link problem.

The paper also dissects the two-stage training pipeline. Comparing the full model against a variant without pretraining, pretraining lifts H@1 by 22.83 points on F1, 12.31 on F2, 7.86 on F3 and 6.27 on F4, and by 1.06 to 10.52 points across N1-N4, evidence that structural patterns captured from the source graph genuinely help the target graph. Fine-tuning matters too: removing it costs between 0.86 and 43.50 H@1 points depending on the dataset. On the practical side, pretraining VMCL takes roughly two to six times longer than MorsE but typically finishes within one to two hours, fine-tuning completes within tens to a few hundred seconds, and per-query inference stays below 0.01 seconds, with the improved variants keeping fine-tuning under 1000 seconds.

The authors are candid about limits. Their setting assumes a fixed relational schema, handling only new entities and not unknown relations; fully inductive scenarios with both new entities and unseen relations remain future work. They also note that very high entity-to-relation ratios, such as the 1098.0 seen on W3, homogenize entity initialization and can degrade performance, which explains the weaker two-stage results on W1-W4. Even so, the combination of explicit contrastive transfer, generative augmentation and a documented scaling study offers the field something it has lacked: evidence that inductive knowledge graph learning improves steadily, and predictably, as capacity grows. The code and datasets have been released on GitHub for other researchers to build on.

Subject of Research: Inductive knowledge graph embedding with contrastive learning and generated representations

Article Title: Contrastive Learning with Growing Generated Representations for Inductive Knowledge Graph Embedding

Article References: Li, Q., Cui, L., Joty, S., Wang, D., Feng, S., Zhang, Y., & Qin, C. (2026). Contrastive Learning with Growing Generated Representations for Inductive Knowledge Graph Embedding. Data Science and Engineering. https://doi.org/10.1007/s41019-026-00352-y

Image Credits: AI Generated

DOI: 10.1007/s41019-026-00352-y

Keywords: knowledge graphs, contrastive learning, variational autoencoder, inductive learning, graph neural networks, link prediction, scaling behavior, machine learning, representation learning, meta-learning, sparsity, Data Science and Engineering