FusionSeg: Topic segmentation model based on semantic feature fusion

doi:10.3969/j.issn.1000-5641.2026.04.013

Abstract

Abstract:

The task of topic segmentation aims to automatically divide text into non-overlapping segments with consistent topics. Traditional topic segmentation models simply map sentences to labels, neglecting the capture of sentence contextual relevance. FusionSeg’s fusion feature encoder dynamically integrates current information with historical information through weights extracted by self-attention mechanisms. Meanwhile, through a combination with fusion contrastive learning, which redefines the positive and negative samples for the topic segmentation task, FusionSeg enhances the role of sentence contextual relevance in the topic segmentation model. In an experimental setup equipped with a single NVIDIA 4090 graphics card, FusionSeg demonstrated a significant improvement in evaluation metric scores, ranging from 1% to 17%, compared to multiple second-best models, such as SeqModel and PEN-NS, across three benchmark datasets: Wiki-727k, En-city, and Wiki-zh. Ablation experiments further demonstrated that FusionSeg’s fusion feature encoder and fusion contrastive learning can capture sentence contextual relevance and optimize topic segmentation results.

Key words: contrastive learning, topic segmentation, feature fusion, self-attention mechanism

CLC Number:

TP391

Zihao GONG, Zhiyun CHEN. FusionSeg: Topic segmentation model based on semantic feature fusion[J]. J* E* C* N* U* N* S*, 2026, 2026(4): 123-133.

Figures/Tables 8

Fig.1

Fig.2

Fig.3

Table 1

Table 2

Table 3

Table 4

Table 5

References 22

1	Hearst M A. Multi-paragraph segmentation of expository text [C]//32nd Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 1994: 9-16.
2	Artemiev A, Parinov D, Grishanov A, et al. Leveraging summarization for unsupervised dialogue topic segmentation [C]//Findings of the Association for Computational Linguistics: NAACL 2024. Association for Computational Linguistics, 2024: 4697-4704.
3	Ma X B, Xu Y, Zhao H, et al.. Multi-turn dialogue comprehension from a topic-aware perspective. Neurocomputing, 2024, 578, 127385.
4	Zhang Q L, Chen Q, Li Y L, et al. Sequence model with self-adaptive sliding window for efficient spoken document segmentation [C]//2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU). IEEE, 2021: 411-418.
5	Gklezakos D C, Misiak T, Bishop D. TreeSeg: hierarchical topic segmentation of large transcripts [PP/OL]. arXiv (2024-06-28)[2024-11-12]. https://doi.org/10.48550/arXiv.2407.12028.
6	Glavaš G, Nanni F, Ponzetto S P. Unsupervised text segmentation using semantic relatedness graphs [C]//Proceedings of the Fifth Joint Conference on Lexical and Computational Semantics. Association for Computational Linguistics, 2016: 125-130.
7	Hearst M A.. Text tiling: segmenting text into multi-paragraph subtopic passages. Computational Linguistics, 1997, 23 (1): 33- 64.
8	Koshorek O, Cohen A, Mor N, et al. Text segmentation as a supervised learning task [C]//Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers). Association for Computational Linguistics, 2018: 469-473.
9	Devlin J, Chang M W, Lee K, et al. BERT: pre-training of deep bidirectional transformers for language understanding [C]//Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). Association for Computational Linguistics, 2019: 4171-4186.
10	Lukasik M, Dadachev B, Papineni K, et al. Text segmentation by cross segment attention [C]//Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). ACL, 2020: 4707-4716.
11	Inan H, Rungta R, Mehdad Y. Structured summarization: unified text segmentation and segment labeling as a generation task [PP/OL]. arXiv (2022-09-28)[2024-11-13]. https://doi.org/10.48550/arXiv.2209.13759.
12	Du L, Buntine W, Johnson M. Topic segmentation with a structured topic model [C]//Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Association for Computational Linguistics, 2013: 190-200.
13	Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need [C]//Proceedings of the 31st International Conference on Neural Information Processing Systems. ACM, 2017: 6000-6010.
14	Lo K, Jin Y, Tan W C, et al. Transformer over pre-trained transformer for neural text segmentation with enhanced topic coherence [C]//Findings of the Association for Computational Linguistics: EMNLP 2021. Association for Computational Linguistics, 2021: 3334-3340.
15	Yu H, Deng C, Zhang Q L, et al. Improving long document topic segmentation models with enhanced coherence modeling [C]//Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 2023: 5592-5605.
16	Liang P P, Zadeh A, Morency L P.. Foundations & trends in multimodal machine learning: principles, challenges, and open questions. ACM Computing Surveys, 2024, 56 (10): 1- 42.
17	Zeng X K, Li Z C, Zhang L F, et al. A coin has two sides: a novel detector-corrector framework for Chinese spelling correction [PP/OL]. arXiv (2024-09-06)[2024-11-21]. https://doi.org/10.48550/arXiv.2409.04150.
18	陈宇航, 王世宙, 汤正婷, 等.. 基于代码和描述文本相融合的软件分类研究. 华东师范大学学报(自然科学版), 2025 (1): 46- 58.
19	Arnold S, Schneider R, Cudré-Mauroux P, et al.. SECTOR: a neural model for coherent topic segmentation and classification. Transactions of the Association for Computational Linguistics, 2019, 7, 169- 184.
20	Barrow J, Jain R, Morariu V, et al. A joint model for document segmentation and segment labeling [C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 2020: 313-322.
21	Gong Z, Tong S W, Wu H, et al. Tipster: a topic-guided language model for topic-aware text segmentation [C]//Bhattacharya A, Lee Mong Li J, Agrawal D, et al. Database Systems for Advanced Applications. Cham: Springer, 2022: 213-221.
22	Xia J X, Liu C, Chen J S, et al. Dialogue topic segmentation via parallel extraction network with neighbor smoothing [C]//Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, 2022: 2126-2131.

数据集名称	文档总数/篇	平均句子数/(句/篇)	句平均字符数/(个/句)	单文档平均主题数/(个/篇)	单主题平均句子数/(句/个)
Wiki-727k 训练集^[8]	582160	47.9	20.7	5.3	9.1
Wiki-727k 开发集^[8]	72354	48.1	20.8	5.3	9.1
Wiki-727k 测试集^[8]	73232	48.7	20.7	5.3	9.1
En-city 数据集^[19]	19500	56.5	/	8.3	6.8
Wiki-zh 训练集^[4]	656602	11.9	38.8	4.7	2.5
Wiki-zh 开发集^[4]	82011	11.8	38.7	4.7	2.5
Wiki-zh 测试集^[4]	82160	11.9	38.8	4.7	2.5

模型名称	Wiki-zh上的实验结果			Wiki-727k上的实验结果
模型名称	$ {P} $	$ {R} $	$ {{F}}_{{1}} $	$ {P} $	$ {R} $	$ {{F}}_{{1}} $
SeqModel:BERT-Base^[4]	0.7840	0.6950	0.7370	0.7060	0.6590	0.6820
Hier.BERT (24-Layers)^[4]	/	/	/	0.6980	0.6350	0.6650
SeqModel:StructBERT-Base^[4]	0.7920	0.7270	0.7580	/	/	/
Cross-segment BERT-Base 128-128^[10]	0.6120	0.8020	0.6940	/	/	0.6400
Cross-segment BERT-Large 128-128^[10]	/	/	/	0.6910	0.6320	0.6600
Cross-segment BERT-Large 256-256^[10]	/	/	/	0.6150	0.7390	0.6710
SeqModel:RoBERTa-Base^[4]	0.7460	0.7370	0.7420	/	/	/
SeqModel:ELECTRA-Base^[4]	0.7350	0.7660	0.7500	/	/	/
FusionSeg	0.8360	0.8340	0.8350	0.8220	0.6980	0.7550

模型名称	En-city上的实验结果
模型名称	$ {{p}}_{{k}} $	$ {{w}}_{{d}} $	$ {{F}}_{{1}} $
SEC>T + bloom^[19]	14.4	/	0.7160
S-LSTM ^[20]	9.10	/	0.7610
BERT-Base + BiLSTM^[20]	9.60	12.80	0.8150
$ \mathrm{Transformer}_{\mathrm{BERT}}^2 $^[14]	9.10	/	/
Tipster^[21]	8.30	/	0.7980
PEN-NS^[22]	8.00	/	0.8000
Naive LongT5-Base-DS^[11]	8.20	/	0.6600
Naive LongT5-Base-SS^[11]	9.20	/	0.7310
BERT-Base ^[15]	8.94	11.34	0.7899
BERT-Base + TSSP + CSSL^[15]	8.22	10.19	0.8020
FusionSeg	7.31	9.94	0.8647

模块/方法名称	En-city上的实验结果			Wiki-727k上的实验结果
模块/方法名称	$ {{p}}_{{k}} $	$ {{w}}_{{d}} $	$ {{F}}_{{1}} $	$ {P} $	$ {R} $	$ {{F}}_{{1}} $
BERT-Base	8.94	11.34	0.7899	/	/	/
BERT-Base + BiLSTM	9.60	12.80	0.8150	0.6730	0.5390	0.5990
BERT-Base + BiLSTM + Fusion Encoder (without attention)	8.63	11.71	0.8356	/	/	/
BERT-Base + BiLSTM + Fusion Encoder	8.62	11.61	0.8371	0.8100	0.6850	0.7420
BERT-Base + BiLSTM + Fusion Encoder (without attention) + FCLL	7.40	10.00	0.8638	/	/	/
BERT-Base + BiLSTM + Fusion Encoder + FCLL	7.31	9.94	0.8647	0.8220	0.6980	0.7550

$ {{\alpha }}_{{2}} $的数值	En-city上的实验结果			Wiki-727k上的实验结果
$ {{\alpha }}_{{2}} $的数值	$ {{p}}_{{k}} $	$ {{w}}_{{d}} $	$ {{F}}_{{1}} $	$ {P} $	$ {R} $	$ {{F}}_{{1}} $
0	8.62	11.61	0.8371	0.8102	0.6851	0.7424
0.5	7.40	10.01	0.8634	0.8171	0.6813	0.7430
0.75	7.56	10.37	0.8669	0.8219	0.6982	0.7550
1	7.49	10.37	0.8574	0.8283	0.6845	0.7496