A unified framework that bridges text-only and multimodal dialogue discourse parsing within a single model has achieved significant average improvements over existing benchmarks in an experimental study. Developed by Doc. Gong Chen, Yu Nan, and Professor Fu Guohong from Soochow University in China, the architecture demonstrates generalizability and effectiveness across different dialogue modalities.
Dialogue discourse parsing is designed to capture the relationships between utterances, facilitating a deeper understanding of dialogue structures and semantics, especially in long and complex dialogues. Existing research has often developed separate dialogue discourse parsers for text-only and multimodal scenarios, largely due to the scarcity of parallel multimodal annotated datasets. This separation limits the ability to fully utilize diverse data with different modalities and poses challenges for real-world artificial intelligence applications.
To solve these problems, the research team proposed a unified framework that bridges text-only and multimodal parsing within a single model. The authors introduced a two-stage training method for the framework inspired by the success of training multimodal large language models. In the first stage, the researchers develop a basic textual dialogue discourse parser and pre-train it on text-only datasets.
In the second stage, the team extends the model to accommodate multimodal scenarios by introducing a multimodal utterance representation module and a multimodal fusion module. The parameters learned from the first stage serve as initial parameters for the second stage and are kept frozen while the multimodal parameters are trained on multimodal datasets.
The researchers conducted extensive experiments on three datasets covering both text-only and multimodal dialogues. The evaluation showed that the proposed approach achieves significant average improvements over several existing benchmarks, confirming the effectiveness of the unified framework across diverse modalities.
The peer-reviewed experimental study was published on 15 September 2026 in Frontiers of Computer Science, co-published by Higher Education Press and Springer Nature.
Subject of Research: Unified framework for textual and multimodal dialogue discourse parsing
Article Title: Bridging modalities: a unified framework for textual and multimodal dialogue discourse parsing
Article References: Bridging modalities: a unified framework for textual and multimodal dialogue discourse parsing. Original publication (Bibliographic details unavailable.)
Image Credits: AI Generated
DOI: Not provided
Keywords: dialogue discourse parsing, multimodal dialogue, text-only dialogue, multimodal fusion module, multimodal utterance representation, two-stage training, Frontiers of Computer Science, Soochow University, Higher Education Press, Springer Nature, multimodal large language models, dialogue structures, dialogue semantics, natural language processing, artificial intelligence, parameter freezing, multimodal annotated datasets, benchmark evaluation
News Source: Denise Maddox. (November 14, 2026). Unified Model Improves Text and Multimodal Dialogue Parsing Across Benchmarks. Scienmag.



