Elamur Research⌕
☰
home / papers / 2026 / 大模型文本简化的篇章结构变化分析与评测

大模型文本简化的篇章结构变化分析与评测

Summary

This study examines how a large language model, GPT-4o, changes the discourse organization of Chinese news articles when it simplifies them. Most existing simplification work evaluates only words and syntax, and assessments of LLMs rely mostly on subjective human ratings. The authors use Rhetorical Structure Theory (RST) to compare three versions of each article: the original, an expert simplification, and a GPT-4o rewrite. They describe the result as the first Chinese parallel discourse-simplification treebank. Each sentence is treated as one EDU, and annotators assigned nucleus–satellite labels and semantic relations. The authors also define nine quantitative indicators covering tree shape and relation properties. Statistical comparison shows that the model's discourse trees are flatter than the experts'. GPT-4o also tends to delete and merge content too aggressively, which causes it to lose core information and weakens referential links. Experts, by contrast, add transitions and restructure the text. The authors propose using these structural indicators for evaluation, structure-aware prompting and model self-correction.

Key points

  • Data: The base corpus is CNS (Chinese News Simplification Data), with 50 originals and 50 expert simplifications drawn from textbooks for beginner learners. The authors added 50 GPT-4o outputs (temperature 0.7, top-p 1.0), giving 150 texts in total.

  • Annotation:

    • The relation set has 34 types, extended from classic RST with several new categories.
    • Annotation was done in RSTTool 3.42 and exported as .rs3 files.
    • Two annotators reached Cohen's Kappa ≈ 0.82.
    • The treebank contains about 6,250 sentence nodes and 3,162 relations, with an average tree depth of about 4.8.

  • Relation counts:

    VersionRelation typesRelation instances
    Originals341,465
    Expert simplifications33931
    GPT-4o32794

  • Nine indicators:

    • Tree-level: maximum depth, mean depth, depth variance, shallow-sentence ratio, sentences per level.
    • Relation-level: nucleus count, satellite count, relation diversity, logical coherence.

  • Statistics: The authors used Kruskal-Wallis H tests with Dunn's post hoc comparisons, and ANOVA for relation diversity. Eight of the nine indicators differed significantly across the three versions; logical coherence was the exception. The authors attribute GPT-4o's weaknesses to four causes: it misidentifies core units, has weak sensitivity to cohesion, over-relies on coordination, and does not compensate for content it removes.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages · Natural language processing

Topics: Discourse analysis

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@article{2026,
  title = {{大模型文本简化的篇章结构变化分析与评测}},
  author = {朱君辉 and 杨尔弘 and 朱浩楠 and 杨麟儿},
  journal = {中文信息学报 (Journal of Chinese Information Processing)},
  year = {2026},
  url = {http://jcip.cipsc.org.cn/article/id/2bc4b900-9adf-4f19-a8c5-5fcb6b62d85e}
}

References

16 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. [ 无障 碍 社 会 的 语 言 之 功 [ 语 言 战 略 研 究, 1] 李宇明 . J]. , ( ) : 2022 7 5 5 - 6. find
  2. [ 2] S IDDHARTHAN A.Pr e s e r v i ng d i s c ou r s es t r uc t u r e //Pr whens imp l i f i ngt ex t[ C] o c e ed i ngso ft he9 t hEu - y r ope an Wo r kshopon Na t u r a lLanguage Gene r a t i ona t EACL,2003. find
  3. [ 3] CROSSLEY S A,LOUWERSE M M,MCCARTHY ,Al P M,e ta l. i ngu i s t i cana l s i so fs imp l i f i edandau - y t hen t i ct ex t s[ J].The Mode r n Language J ou r na l, 2007,91( 1): 15 - 30. find
  4. [ ] , 4 ARFÉB MASONL,FAJARDOI. S imp l i f i ngi n f o r - y ma t i ona lt ex ts t r uc t u r ef o rs t r ugg l i ng r e ade r s[ J]. Re ad i ngand Wr i t i ng,2018,31( 9):2191 - 2210. find
  5. [ ] , , 5 XU W CALLI SON BURCH C NAPOLESC.Pr ob - l emsi ncu r r en tt ex ts imp l i f i c a t i onr e s e a r ch:Newda t a c anhe l J].Tr ans a c t i onso ft heAs s o c i a t i onf o rCom-p[ t a t i ona lL i ngu i s t i c s,2015( 3):283 - 297. pu [ ,Imp 6] PATEL N,NAGPALP,SHAH T,e ta l. r ov i ng :… find
  6. [ 7] ZHONG Y,J IANG C,XU W,e ta l.Di s c ou r s el e ve l f a c t o r sf o rs en t enc e de l e t i on i nt ex ts imp l i f i c a t i on [ //Pr C] o c e ed i ngso ft heAAAICon f e r enc eon Ar t i f i - c i a lI n t e l l i e,2020( 5):9709 - 9716. genc [ 8] RETSI,ASTRUCL,COUGHLAN T,e ta l. App r oa… find
  7. [ 9] 吴继峰 . 多元视角下 留 学 生 汉 语 教 材 文 本 改 编 策 略 研 究[ J]. 国际汉语教学研究,2024( 2):62 - 73. find
  8. [ ] 10 高雪松,周小兵,洪 炜 . 国 际 中 文 教 材 文 本 改 编: 基 本原则与实现路径[ J]. 四川师范大学学报( 社会 科 学 , ( ) : 版) 2024 4 138 - 144. find
  9. [ 11] MARTÍNEZ P,RAMOS A,MORENO L.Exp l o - r i ngl a r anguage mode l st ogene r a t ee a syt or e ad gel 朱君辉( 1998—), 博 士, 主 要 研 究 领 域 为 自 然 语 言处理和智能语言学习。 E-ma i l: nys zxz h@163. c om y j 朱浩楠( 2001—), 硕 士, 主 要 研 究 领 域 为 自 然 语 言处理和智能语言学习。 E-ma i l: 1930940919@qq. c om c on t en t[ J].Fr on t i e r… find
  10. [ 大语言模型的中文文本 12] 杨尔 弘, 朱 君 辉, 朱 浩 楠, 等 . 简化能力研究[ 语言战略研究,2024, J]. 9( 5):34 - 47. find
  11. [ 13] LIU F,J IANG Y,LAIC,e ta l. Te a che rengagemen t wi t hau t oma t edt ex ts imp l i f i c a t i onf o rd i f f e r en t i a t ed i ns t r uc t i on[ J].Language Le a r n i ng & Te chno l ogy, 2024, 28( 2):163 - 182. find
  12. [ 14] 娄开阳 . 现代汉语新闻语篇的结构研究[M]. 北京: 世 界图书出版公司,2008. find
  13. [ 15] MANN W C,THOMPSONSA.Re l a t i ona lp r opo s i - t i onsi nd i s c ou r s e[ J].Di s c ou r s ePr o c e s s e s,1986, 9 ( 1):57 - 90. find
  14. [ 16] VAN DI JK T A,KINTSCH W.S t r a t eg i e so fd i s - c ou r s ec omp r ehens i on [M ].New Yo r k: Ac ademi c Pr e s s,1983. find
  15. [ 17] CRI STEA D,IDEN,ROMARYL.Ve i nst he o r y:A mode lo fg l oba ld i s c ou r s ec ohe s i on and c ohe r enc e [ //Pr C] o c e ed i ngso ft heCOLING-ACL Con f e r enc e, 1998. find
  16. [ 基于主题相似度的宏观篇章 18] 蒋峰, 褚 晓 敏, 徐 昇, 等 . 主次关系 识 别 方 法 [ 中 文 信 息 学 报, J]. 2018, 32( 1): find

↑ back to top

This page in Markdown →