Elamur Research⌕
☰
home / papers / 2023 / DiscoFlan: Instruction Fine-tuning and Refined Text Generation for Discourse Relation Label Classification

DiscoFlan: Instruction Fine-tuning and Refined Text Generation for Discourse Relation Label Classification

Summary

This DISRPT 2023 shared-task submission casts discourse relation classification (DRC) as text generation rather than standard categorical prediction. The author fine-tunes Flan-T5 (the small checkpoint, google/flan-t5-small) on sentence pairs wrapped in instruction prompts translated into each dataset's language. The model then writes out the relation label as a short string. Raw generations often fall outside the valid label set (for example "elaborated" instead of "elaboration"), so a refinement step is added. It strips affixes, matches the remaining lemma against the training labels, and falls back to the majority label when no match is found. This variant is called DiscoFlan+Ref. The approach needs no hand-crafted features, unlike the previous best system DiscoDisco, and covers all 26 shared-task datasets across RST, PDTB, SDRT and dependency formalisms. The results are modest. Refinement is essential, since unrefined output scores 0 on most corpora. Even so, DiscoFlan+Ref stays well below DiscoDisco and frequently below an Xlm-R baseline. The work matters mainly as an early exploration of generative, multilingual, instruction-based DRC.

Key points

  • The benchmark has 26 DISRPT datasets, 11 of them new in 2023, with label inventories ranging from 9 (zho.pdtb.cdtb) to 33 (nld.rst.nldt).
  • The setup used FlanT5-small with batch size 16 and up to 50 epochs with early-stopping patience 12. The learning rate was 1e-3 for small datasets and 1e-5 for large ones. Decoding used beam width 4 with per-dataset min/max generation lengths, on a 32 GB V100.
  • Accuracy for DiscoFlan+Ref includes fas.rst.prstc at 45.44, against 52.53 for DiscoDisco with features and 34.01 for Xlm-R. Other scores are zho.rst.gcdt at 59.37 and eng.pdtb.pdtb at 24.41, where DiscoDisco reaches 74.44 and Xlm-R 66.95.
  • The author reports that the model tends to overfit on a single label. They also acknowledge that the majority-label fallback assumes skewed label distributions.
  • Code is released at github.com/erzaliator/DiscoFlan. Larger models such as FlanT5-large are left for future work.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Discourse analysis

Cited by

11 papers of our library cite it (of the 140673 whose bibliographies we read); 1 of them have a page here, 10 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2023.disrpt-1.2

@inproceedings{anuranjana2023discoflan,
  title = {{DiscoFlan: Instruction Fine-tuning and Refined Text Generation for Discourse Relation Label Classification}},
  author = {Anuranjana, Kaveri},
  booktitle = {Proceedings of the 3rd Shared Task on Discourse Relation Parsing and Treebanking (DISRPT 2023)},
  year = {2023},
  doi = {10.18653/v1/2023.disrpt-1.2},
  url = {https://doi.org/10.18653/v1/2023.disrpt-1.2}
}

References

11 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Lynn Carlson, Mary Ellen Okurowski, and Daniel Marcu. 2002. RST discourse treebank. Linguistic Data Consortium, University of Pennsylvania. find
  2. Rashmi Prasad, Nikhil Dinesh, Alan Lee, Eleni Milt-sakaki, Livio Robaldo, Aravind Joshi, and Bonnie Webber. 2008. The Penn Discourse TreeBank 2.0. In Proceedings of the Sixth International Conference on Language Resources and Evaluation (LREC’08), Marrakech, Morocco. European Language Resources Asso… find our card
  3. Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Eric Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, et al. 2022. Scaling instruction-finetuned language models. arXiv preprint arXiv:2210.11416. find
  4. Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971. find
  5. Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2020. Unsupervised cross-lingual representation learning at scale. In Proceedings of the 58th Annual Meeting of the Association fo… find
  6. Frances Yung, Kaveri Anuranjana, Merel Scholman, and Vera Demberg. 2022. Label distributions help implicit discourse relation classification. In Proceedings of the 3rd Workshop on Computational Approaches to Discourse, pages 48–53. find our card
  7. Luke Gessler, Shabnam Behzad, Yang Janet Liu, Siyao Peng, Yilun Zhu, and Amir Zeldes. 2021. Dis-CoDisCo at the DISRPT2021 shared task: A system for discourse segmentation, classification, and connective detection. In Proceedings of the 2nd Shared Task on Discourse Relation Parsing and Treebanking (D… find
  8. Amir Zeldes, Yang Janet Liu, Mikel Iruskieta, Philippe Muller, Chloé Braud, and Sonia Badene. 2021. The disrpt 2021 shared task on elementary discourse unit segmentation, connective detection, and relation classification. In Proceedings of the 2nd Shared Task on Discourse Relation Parsing and Treeba… find our card
  9. Yuxin Jiang, Linhan Zhang, and Wei Wang. 2022. Global and local hierarchy-aware contrastive framework for implicit discourse relation recognition. arXiv preprint arXiv:2211.13873. find our card
  10. Di Jin, Zhijing Jin, Joey Tianyi Zhou, and Peter Szolovits. 2020. Is bert really robust? a strong baseline for natural language attack on text classification and entailment. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 8018–8025. find our card
  11. Mihir Kale and Abhinav Rastogi. 2020. Text-to-text pre-training for data-to-text tasks. In Proceedings of find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/anuranjana-2023-discoflan-instruction-fine-tuning-refined/