文本结构划分是自动文摘研究中的一个关键阶段,也是自然语言处理领域的重要课题.本文通过构建段落向量空间模型,提出一种综合考察相邻段落相似度和连续段落平均相似度的意义段划分方法,使文摘内容更加全面,结构更加平衡.实验结果表明,该方法能够较有效地反映文章的内容结构,对有子标题组织和无子标题组织的文章均适用;由于考虑了总起段,使得文本结构划分更加合理,为自动文摘系统的后续工作打下坚实的基础.
Text structure partition is a significant stage in the automatic text summarization as well as an important issue in nature language processing. The topic partition is based on vector space model (VSM) in this paper. Different from the existing approaches that make use of the similarity of adjacent paragraphs, we put forward an algorithm for topic partition based on a comprehensive investigation of both adjacent paragraphic similarity and consecutive average paragraphic similarity. This makes the summarization more comprehensive in content and more balanced in structure. At the same time, the topic number of article is determined automatically. We also find that the importance of recapitulative paragraph is neglected by the previous investigations and propose a method to recognize the recapitulative paragraph. This makes the topic partition more reasonable. We designate three experiments as the basis of topic partition: the adjacent paragraphic similarity, the consecutive paragraphic similarity, and the comprehensive investigation of both adiacent paragraphic similarity and consecutive average paragraphic similarity. The result shows that the method we put forward is superior to the previous methods. The result also shows that the headlines contribute to topic partition, whereas our approach is also suitable for the topicpartition of the articles without headlines. This lays a good foundation for the research of the automatic text summarization system.