Knowledge graphs steer diffusion AI to rebuild Huizhou dwelling facades

Generative image models can now paint almost anything, but painting a building that a heritage architect would actually accept is a much harder problem. A study published in the journal Architectural Intelligence on 14 September 2026 argues that the missing ingredient for regional architectural styles is not more pixels or bigger models, but explicit construction knowledge, encoded in a graph and fed directly into the diffusion process. The research team, led by Zhe Guo with Yuhang Kong, Zihuan Zhang and Qingyi Xiong, developed a framework called the Knowledge-Augmented Stable Diffusion Model, or KA-SDM, and tested it on the facades of Huizhou traditional dwellings, a vernacular architecture type from southern Anhui Province in China.

The problem the authors set out to solve is well known to anyone who has tried to coax a diffusion model into reproducing a specific building tradition. Prompt engineering or standard fine-tuning alone rarely encodes the construction logic, spatial hierarchy and symbolic rules that make such heritage architecture intelligible. A generated image may look stylistically plausible at first glance, yet place windows where structural logic forbids them, misread the rhythm of the timber-frame bays, or attach decorative motifs to the wrong part of the wall. For Huizhou dwellings, whose facade forms depend on typological relations among courtyard layouts, timber-frame bay rhythms, materials and decorative motifs, this gap between visual similarity and semantic correctness is particularly acute.

The framework attacks the problem in three layers. First, the researchers organized construction knowledge into a property graph schema inspired by the CIDOC Conceptual Reference Model, the reference ontology used in cultural heritage informatics for semantic interoperability. The schema, implemented in the Neo4j graph database, represents posts, spaces, ratios, wall plans, elements and details as queryable entities and relations. The authors are careful about scope here: their HzTD schema is a task-oriented adaptation for facade generation, and full CIDOC-CRM compliance and OWL-style ontology reasoning lie outside the study. Second, rule-based graph inference derives facade composition attributes, opening configurations and detail semantics from limited plan-level information. Third, a Prompt-Conditioned Semantic Controller converts those inferred attributes into structured prompt fragments and ControlNet masks that steer the image generator.

The knowledge representation itself is deliberately architectural rather than generic. The schema groups Huizhou dwelling attributes into four families: Function and Type, covering space types; Layout and Composition, covering plan and facade elements; Material and Tectonic, recording features such as white-rendered walls, grey tiles, timber components, grey brick and brick relief; and Culture and Symbolism, covering motifs, ritual hierarchy, narrative meaning and regional style. Huizhou dwellings, which originated during the Song dynasty and flourished in the Ming dynasty, centre on an open-air courtyard known in the Huizhou context as TianJing, with front and rear halls arranged along a central axis in a front-hall, rear-chamber axial layout. The buildings use timber-frame construction with locally sourced timber, stone, grey bricks and tiles, enclosed courtyards surrounded by high walls on all four sides, and the iconic white walls and grey tiles aesthetic, crowned by the distinctive horse-head gable walls, or MaTouQiang, that articulate the roofline.

The graph decomposes each dwelling into a three-level entity structure. At the top, PlanLayout and FacadeComposition link to entities such as Interface, Building, Space, Post, Ratio, WallPlan, Detail and Element, each specialized into architectural sub-types. The Interface node distinguishes external and internal facades; the Space node covers rooms such as Tang, Ting, CeShi, Yuan and Lang; and the Post node encodes structural members including LangZhu, BuZhu, JiZhu and MenZhu. Quantitative and geometric aspects are captured through Ratio properties such as Kaijian and JinShen, the modular horizontal and longitudinal spans of the timber frame that structure the rhythm, hierarchy and proportional order of the spatial organization. Four reusable relationship types, RELATIVE_TO, PART_OF, HAS and IS, tie the entities together so that geometric, structural, material and symbolic information coexist in one queryable graph.

Rule-based inference then turns this stored knowledge into facade knowledge through three complementary patterns. Component-level spatial logic inference uses geometric attributes such as post coordinates to recover topological relations between spaces, for example creating a NEXT_TO relationship between Tang and Lang spaces whose posts share aligned x-coordinates. Class-hierarchy-based structural logic inference propagates semantic labels from space types to facade composition units, so a Tang space can pass its label to a corresponding WallPlan node through PART_OF and IS relationships. Attribute and label-based logic inference applies declarative rules to generate composite facade types, such as a WallPlan node typed Ting-Tang at the front position when a front interface co-occurs with Ting and Tang spaces. Applied iteratively, these rules reconstruct spatial adjacency from geometric components, lift space types to facade units and compose them into facade patterns, all exported as structured CSV inputs for the generation stage.

The generation stage conditions a Stable Diffusion backbone in two ways. Graph-derived data are mapped by a Python script onto a 512 by 512 pixel canvas through coordinate transformation, producing semantic segmentation masks without any 3D modelling, which the authors say significantly improves generation efficiency by focusing solely on pixel-level mask outputs. Meanwhile, the Prompt-Conditioned Semantic Controller works in three stages: a semantic encoder queries the property graph and encodes the current context into a structured attribute tuple; a rule-mapping layer looks up a predefined JSON rule table to select the corresponding building configuration and LoRA presets; and a prompt composer converts the semantic configuration into a detail-level prompt snippet concatenated with the global facade prompt. In the experimental case, a tuple specifying function as dwelling, period as Qing, wall types SanKaiJian and MaTouQiang and detail type BrickRelief maps to material descriptions of grey brick and white lime paint, shallow carving style, landscape and prosperity motifs, auspicious and domesticity symbolism, and a weighted LoRA trigger for the MaTouQiang wall type.

For the case study, the team selected a representative two-bay dwelling of the most typical type in Xidi Village as the spatial prototype, deliberately starting from minimal plan-level information attached to the post grid to stress-test the inference pipeline. The LoRA fine-tuning dataset contained 60 curated Huizhou dwelling facade images compiled from field photographs and publicly available architectural photography, with watermarked, occluded, low-resolution or weakly relevant images excluded, all pre-processed to 512 by 512 pixel PNG format. The base model was the open-source Stable Diffusion v1.5, chosen for its mature LoRA and ControlNet ecosystem and its practicality for small-sample heritage datasets compared with larger backbones such as SDXL. The LoRA configuration used a low-rank decomposition with r equal to 2 to mitigate overfitting, a network dimension of 64, 15 training epochs at batch size 1, roughly 400 optimization steps, and trigger tags binding the model to Huizhou dwelling facades in alleyway contexts.

Evaluation compared the full KA-SDM against a plain Stable Diffusion baseline and three ablation settings: Stable Diffusion plus LoRA, plus LoRA and generic ControlNet, and plus LoRA and graph-derived ControlNet masks. The researchers sampled 20 front-facade images from each model, shuffled the resulting 100 samples and had five experts in architectural heritage and Huizhou vernacular architecture rate each image on a one-to-five scale across four criteria: typological fidelity, cultural symbolism expression, spatial composition logic and material construction plausibility. Inter-rater agreement, analysed with Cohen’s kappa, was 0.76, indicating substantial agreement. The authors also introduced a composite Expert-Rated Facade Plausibility index, built from the two core layout dimensions, and calculated Fréchet Inception Distance against the 60-image reference set, treating FID as a relative visual-distribution indicator given the small, case-specific reference set.

The results, reported qualitatively and through the expert metrics, indicate that knowledge-guided semantic control improves typological consistency, structural plausibility and expert-rated semantic alignment while preserving visual diversity. The plain baseline produced generic alley scenes with weak or incorrect Huizhou style; adding the LoRA enhanced roof silhouettes but left cluttered, inconsistent facades; generic ControlNet line-art aligned wall outlines but, lacking Huizhou-specific component identity and material semantics, could over-constrain contours and produce texture-semantic inconsistency, which the authors link to its relatively high FID. The full KA-SDM, by contrast, generated facades whose roof profiles agreed with the JinShen ratio parameters, whose door and window proportions matched the KaiJian rules encoded in the graph, and whose brick-relief zone appeared at the correct position above the central door; in the perspective task, eave lines converged correctly toward the vanishing direction. The authors frame the work as a practical pathway for coupling architectural heritage knowledge representation with generative AI, supporting cultural documentation, design assistance and virtual reconstruction, and note that future research will extend the framework to larger-scale settlement patterns, multimodal two-dimensional to three-dimensional generation and interactive virtual heritage applications. The study’s scope remains a single prototypical dwelling type from one village, with variations in spatial layout left for later research stages.

Subject of Research: A knowledge-graph-augmented stable diffusion framework for generating semantically controlled facades of Huizhou traditional dwellings

Article Title: A knowledge-augmented diffusion framework for generating regionally distinctive architectural facades: a case study of Huizhou dwellings

Article References: Guo, Z., Kong, Y., Zhang, Z., & Xiong, Q. (2026). A knowledge-augmented diffusion framework for generating regionally distinctive architectural facades: a case study of Huizhou dwellings. Architectural Intelligence, 5(1), Article 27. https://doi.org/10.1007/s44223-026-00137-3

Image Credits: AI Generated

DOI: 10.1007/s44223-026-00137-3

Keywords: generative AI, diffusion models, knowledge graphs, architectural heritage, Huizhou dwellings, facade generation, Stable Diffusion, ControlNet, LoRA, Neo4j, CIDOC CRM, Xidi Village