Elamur Research⌕
☰
home / papers / 2025 / System Report for CCL25-Eval Task 4: Prompting, Scheduling, and Arbitration Strategies for Chinese Factivity Inference

System Report for CCL25-Eval Task 4: Prompting, Scheduling, and Arbitration Strategies for Chinese Factivity Inference

Summary

The paper describes the Soochow University team's system for the Chinese Frame Semantic Parsing (CFSP) task at CCL25-Eval. The task is based on Chinese FrameNet (CFN) and has three subtasks: frame identification (FI), argument identification (AI) and role identification (RI). This year's evaluation pays particular attention to nested roles. The authors build on the baseline of Li et al. (2023b), which classifies entities using RoPE. They replace chinese-roberta-wwm-ext in it with the larger chinese-roberta-wwm-ext-large model. For FI, they add two features: word segmentation labels in the BMES scheme and fasttext vectors of the target word, which are fine-tuned together with the model. For RI, they use the same features and replace cross-entropy with Focal Loss to reduce the effect of class imbalance. In FI and RI, the predictions of several runs are combined by voting. The system ranked first on TestA and second on TestB. The ablation study showed that the larger model contributes the most. The work matters because it confirms that, for frame semantic parsing, well-tuned compact encoders with linguistic features remain competitive with LLM-based approaches.

Key points

  • On TestB, the system obtained FI accuracy 71.8960, AI F1 85.5192, RI F1 57.6181 and an overall score of 70.2718. In FI it slightly outperformed the winner, while in AI and RI it fell behind.
  • FI requires choosing among 713 frames. TestA contains 75 frames that are absent from the training data, so Focal Loss did not improve FI.
  • For voting, the model was run three times with Dropout enabled and seeds 1, 77 and 777. The γ parameter of Focal Loss was set to 1.
  • Without the large model, performance drops most noticeably in AI (−2.53 F1) and RI (−1.08 F1). Without voting, FI unexpectedly rises to 72.1977, and without Focal Loss, RI rises to 57.9126.
  • The code and data are published on GitHub (yahui19960717/CCL-2025-CFN).

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages · Natural language processing

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{liu2025system,
  title = {{System Report for CCL25-Eval Task 4: Prompting, Scheduling, and Arbitration Strategies for Chinese Factivity Inference}},
  author = {Liu, Daohuan and Xia, Lun and Zhang, Yuxuan and Yang, Xinyu and Kong, Fanzhen},
  booktitle = {Proceedings of the 24th China National Conference on Computational Linguistics (CCL 2025)},
  year = {2025},
  url = {https://aclanthology.org/2025.ccl-2.17/}
}

References

23 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. In Proceedings of the 34th Conference on Neural Information Processing Systems, pages 1877–190… find
  2. Charles J Fillmore, Christopher R Johnson, and Miriam RL Petruck. 2003. Background to framenet. International journal of lexicography, 16(3):235–250. find
  3. Yong Guan, Shaoru Guo, Ru Li, Xiaoli Li, and Hongye Tan. 2021a. Frame semantic-enhanced sentence modeling for sentence-level extractive text summarization. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 4045–4052. find our card
  4. Yong Guan, Shaoru Guo, Ru Li, Xiaoli Li, and Hu Zhang. 2021b. Integrating semantic scenario and word relations for abstractive sentence summarization. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 2522–2529. find
  5. Yingxuan Guan, Xunyuan Liu, Lu Zhang, Zexian Xie, and Binyang Li. 2023. System report for CCL23-eval task 3: UIR-ISC pre-trained language medel for Chinese frame semantic parsing. In Proceedings of the 22nd Chinese National Conference on Computational Linguistics (Volume 3: Evaluations), pages 124–1… find
  6. Shaoru Guo, Yong Guan, Ru Li, Xiaoli Li, and Hongye Tan. 2020a. Incorporating syntax and frame semantics in neural network for machine reading comprehension. In Proceedings of the 28th International Conference on Computational Linguistics, pages 2635–2641. find our card
  7. Shaoru Guo, Ru Li, Hongye Tan, Xiaoli Li, Yong Guan, Hongyan Zhao, and Yueping Zhang. 2020b. A frame-based sentence representation for machine reading comprehension. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 891–896. find
  8. Shutan Huang, Qiuyan Shao, and Wei Li. 2023. System report for CCL23-eval task 3: Chinese frame semantic parsing based on multi task pipeline strategy. In Proceedings of the 22nd Chinese National Conference on Computational Linguistics (Volume 3: Evaluations), pages 105–112. find
  9. Juncai Li, Zhichao Yan, Xuefeng Su, Boxiang Ma, Peiyuan Yang1, and Ru Li. 2023a. Overview of CCL23-eval task 1:Chinese FrameNet semantic parsing. In Proceedings of the 22nd Chinese National Conference on Computational Linguistics (Volume 3: Evaluations), pages 113–123. find
  10. Zuoheng Li, Xuanzhi Guo, Dengjian Qiao, and Fan Wu. 2023b. System report for CCL23-eval task 3: Application of entity classification model based on rotary position embedding in chinese frame semantic parsing. In Proceedings of the 22nd Chinese National Conference on Computational Linguistics (Volume… find
  11. Ru Li, Yunxiao Zhao, Zhiqiang Wang, Xuefeng Su, Shaoru Guo, Yong Guan, Xiaoqi Han, and Hongyan Zhao. 2024. A comprehensive overview of cfn from a commonsense perspective. Machine Intelligence Research, 21(3):239–256. find
  12. Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár. 2017. Focal loss for dense object detection. In Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV), pages 2999– 3007. find our card
  13. Yahui Liu, Zhenghua Li, and Min Zhang. 2023. System report for CCL23-eval task3: SUDA CFSP system. In Proceedings of the 22nd Chinese National Conference on Computational Linguistics (Volume 3: Evaluations), pages 84–93. find
  14. Takeru Miyato, Andrew M. Dai, and Ian J. Goodfellow. 2017. Adversarial training methods for semi-supervised text classification. In Proceedings of the 5th International Conference on Learning Representations, pages 1–11. find our card
  15. Yang Peiyuan, Li Juncai, Yan Zhichao, Su Xuefeng, and Ru Li. 2024. Chinese frame semantic parsing evaluation. In Proceedings of the 23rd Chinese National Conference on Computational Linguistics (Volume 3: Evaluations), pages 32–42. find
  16. Xuefeng Su, Ru Li, Xiaoli Li, Jeff Z Pan, Hu Zhang, Qinghua Chai, and Xiaoqi Han. 2021. A knowledge-guided framework for frame identification. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Pr… find
  17. Jianlin Su, Ahmed Murtadha, Shengfeng Pan, Jing Hou, Jun Sun, Wanwei Huang, Bo Wen, and Yunfeng Liu. 2022. Global pointer: Novel efficient span-based approach for named entity recognition. arXiv preprint arXiv:2208.03054. 2 5 find
  18. Xiaofei Sun, Linfeng Dong, Xiaoya Li, Zhen Wan, Shuhe Wang, Tianwei Zhang, Jiwei Li, Fei Cheng, Lingjuan Lyu, Fei Wu, et al. 2023. Pushing the limits of ChatGPT on NLP tasks. arXiv preprint arXiv:2306.09719. find
  19. Xiaohui Wang, Ru Li, Zhiqiang Wang, Qinghua Chai, and Xiaoqi Han. 2020. Syntax-aware Chinese frame semantic role labeling based on self-attention. In Proceedings of the 19th Chinese National Conference on Computational Linguistics, pages 616–623. find
  20. Liu Yahui, Gong Chen, and Zhang Min. 2024. Leveraging LLMs for Chinese frame semantic parsing. In Proceedings of the 23rd Chinese National Conference on Computational Linguistics (Volume 3: Evaluations), pages 21–31. find
  21. Zhichao Yan, Jiapu Wang, Jiaoyan Chen, Xiaoli Li, Ru Li, and Jeff Z Pan. 2024. Atomic fact decomposition helps attributed question answering. arXiv preprint arXiv:2410.16708. — [arXiv:2410.16708](https://arxiv.org/abs/2410.16708) find
  22. Liping You and Kaiying Liu. 2005. Building Chinese Framenet database. In Proceedings of the 2005 international conference on natural language processing and knowledge engineering, pages 301–306. find
  23. Hongyan Zhao, Ru Li, Xiaoli Li, and Hongye Tan. 2020. CFSRE: Context-aware based on frame-semantics for distantly supervised relation extraction. Knowledge-Based Systems, 210(2020):106480. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/liu-2025-system-report-ccl25-eval-task/