Elamur Research⌕
☰
home / papers / 2024 / Exploring Continual Learning of Compositional Generalization in NLI

Exploring Continual Learning of Compositional Generalization in NLI

Paper · 2024 · Xiyan Fu and Anette Frank

Summary

Earlier work on compositional generalization in NLI trains models offline, with every primitive inference available at once. Fu and Frank ask what happens when the primitives arrive one after another, as they do in human learning. They define the C2Gen NLI challenge, in which a model learns primitive inference types in sequence and is then tested on unseen compositions of them. The compositions combine veridical inferences (e.g., try to S vs. manage to S) with ordinary lexical NLI. They build a compositional NLI dataset, re-partition it into a continual stream, and evaluate two subtasks: compositional inference (Task CI) and recognition of the underlying primitives (Task P). A multi-task model trained this way fails to generalize compositionally, mainly because it forgets earlier primitives. Memory-based continual learning algorithms reduce forgetting but do not close the gap. Ordering the stream helps: primitives before compositions, and easier inference types before harder ones. The effect also holds for Llama-2 when data leakage is controlled with pseudo data. The work matters for systems that must reason over knowledge that keeps being updated.

Key points

  • New task: C2Gen NLI (Continual Compositional Generalization in NLI), published in TACL vol. 12 (2024), pp. 912–932. Data and code are at github.com/Heidelberg-NLP/C2Gen.
  • Two evaluation subtasks:
    • Task CI checks whether the model gets the unseen composed inference right.
    • Task P checks whether it still resolves each constituent primitive.
  • Forgetting is the main cause of failure in the continual setting. Data-based continual learning methods, which replay small stored memories of past examples, help, but memorization alone does not solve compositional inference.
  • Learning order matters:
    • Primitive-before-compositional and easy-before-hard schedules move continual learning closer to non-continual performance.
    • For Llama-2, the paper reports a 1.6-point drop under the ver → nat order and a 0.66-point rise under nat → ver. The authors read this as support for an easy-to-hard scheme.
  • Proposed application: Persona Dialogue, where a persona profile grows over time. New facts (e.g., buying a computer) must combine with old ones (a favourite colour) to judge whether an utterance is consistent with the persona.

Interesting ideas

  • I1. Testing a composite inference together with a separate check of each of its parts shows whether a failure comes from forgetting a part or from failing to combine parts.
  • I2. The same data can be re-partitioned into a sequential stream to simulate incremental acquisition, which exposes forgetting that shuffled training hides.
  • I3. Ordering training so that building blocks come first and harder inference types come later works like a curriculum and measurably helps generalization.
  • I4. Veridicality of embedding verbs (try, manage, fail) is treated as a reusable primitive that composes with lexical inference over the embedded clause.
  • I5. Checking persona consistency against a profile that keeps growing means judging entailment or contradiction relative to an updated information state.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (answer): Each QA answer could be paired with probes of the individual frame facts it relies on, so a wrong answer is traced to a missing fact or to a faulty combination step.
  • I2 (learn): When new silver data or repair rules are added in batches, re-run earlier evaluation slices after each batch to catch regressions on older phenomena.
  • I3 (learn): Training of gparse or the readers could be scheduled from simple constructions to embedded or composed ones, with the ordering justified by measured difficulty rather than intuition.
  • I4 (world): Store a veridicality signature per verb class, so the world model can decide whether an embedded event actually happened and combine that with lexical inference such as dog → pet.
  • I5 (world): The text's world state could be updated fact by fact, with each new sentence checked for contradiction against facts composed from earlier ones.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Heidelberg UniversityHeidelberg · Germany38 works · 41,356 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 1Research subjects and languages the work is about.Reasoning · 1,426 works

Cite

Cite as Xiyan Fu and Anette Frank. 2024. Exploring Continual Learning of Compositional Generalization in NLI. Transactions of the Association for Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.1162/tacl_a_00680

@article{fu2024exploring,
  title = {{Exploring Continual Learning of Compositional Generalization in NLI}},
  author = {Xiyan Fu and Anette Frank},
  journal = {Transactions of the Association for Computational Linguistics},
  year = {2024},
  doi = {10.1162/tacl\_a\_00680},
  url = {https://doi.org/10.1162/tacl_a_00680}
}

References

62 references, 26 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Gustavo Aguilar, Yuan Ling, Yu Zhang, Benjamin Yao, Xing Fan, and Chenlei Guo. 2020. Knowledge distillation from internal representations. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 7350–7357. https://doi.org/10.1609/aaai.v34i05.6229 ↗
  2. Magdalena Biesialska, Katarzyna Biesialska, and Marta R. Costa-jussà. 2020. Continual lifelong learning in natural language processing: A survey. In Proceedings of the 28th International Conference on Computational Linguistics, pages 6523–6541, Barcelona, Spain (Online). International Committee on Computational Linguistics. https://doi.org/10 .18653/v1/2020.coling-main.574 ↗
  3. Rahaf Aljundi, Eugene Belilovsky, Tinne Tuytelaars, Laurent Charlin, Massimo Caccia, Min Lin, and Lucas Page-Caccia. 2019. Online continual learning with maximal interfered Samuel R. Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Manning. 2015. A large annotated corpus for learning natural language inference. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 632–642, Lisbon, Portugal. Association for Computational Linguistics. https://doi.org/10 .18653/v1/D15-1075 ↗
  4. Levskaya, Sanjay Ghemawat, Sunipa Dev, Henryk Michalewski, Xavier Garcia, Vedant Misra, Kevin Robinson, Liam Fedus, Denny Zhou, Daphne Ippolito, David Luan, Hyeontaek Lim, Barret Zoph, Alexander Spiridonov, Ryan Sepassi, David Dohan, Shivani Agrawal, Mark Omernick, Andrew M. Dai, Thanumalayan Sankaranarayana Pillai, Marie Pellat, Aitor Lewkowycz, Erica Moreira, Rewon Child, Oleksandr Polozov, Katherine Lee, Zongwei Zhou, Xuezhi Wang, Brennan Saeta, Mark Diaz, Orhan Firat, Michele Catasta, Jason Wei, Kathy Meier-Hellstern, Douglas Eck, Jeff Dean, Slav Petrov, and Noah Fiedel. 2023. Palm: Scaling language modeling with pathways. Journal of Machine Learning Research, 24(240):1–113.
  5. Arslan Chaudhry, Marc’Aurelio Ranzato, Marcus Rohrbach, and Mohamed Elhoseiny. 2019a. Efficient lifelong learning with a-gem. In International Conference on Learning Representations. ↗
  6. Arslan Chaudhry, Marcus Rohrbach, Mohamed Elhoseiny, Thalaiyasingam Ajanthan, Puneet Kumar Dokania, Philip H. S. Torr, and Marc’ Aurelio Ranzato. 2019b. On tiny episodic memories in continual learning. arXiv: Learning. ↗
  7. Alexis Conneau, Ruty Rinott, Guillaume Lample, Adina Williams, Samuel Bowman, Holger Schwenk, and Veselin Stoyanov. 2018. XNLI: Evaluating cross-lingual sentence representations. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2475–2485, Brussels, Belgium. Association for Computational Linguistics. https://doi.org/10 .18653/v1/D18-1269 ↗
  8. Jifan Chen, Eunsol Choi, and Greg Durrett. 2021. Can NLI models verify QA systems’ predictions? In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 3841–3854, Punta Cana, Dominican Republic. Association for Computational Linguistics. https://doi.org/10.18653/v1/2021.findings-emnlp.324 ↗
  9. Ido Dagan, Dan Roth, Mark Sammons, and Fabio Massimo Zanzotto. 2013. Recognizing textual entailment: Models and applications. Synthesis Lectures on Human Language Technologies, 6(4):1–220. https://doi.org/10.1007/978-3-031-02151-0 ↗
  10. Qian Chen, Xiaodan Zhu, Zhen-Hua Ling, Diana Inkpen, and Si Wei. 2018. Neural natural language inference models enhanced with external knowledge. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2406–2417, Melbourne, Australia. Association for Computational Linguistics. https://doi.org/10.18653/v1/P18 -1224 ↗
  11. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics. https://doi.org/10 .18653/v1/N19-1423 Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, Parker Schuh, Kensen Shi, Sasha Tsvyashchenko, Joshua Maynez, Abhishek Rao, Parker Barnes, Yi Tay, Noam Shazeer, Vinodkumar Prabhakaran, Emily Reif, Nan Du, Ben Hutchinson, Reiner Pope, James Bradbury, Jacob Austin, Michael Isard, Guy Gur-Ari, Pengcheng Yin, Toju Duke, Anselm Nouha Dziri, Ximing Lu, Melanie Sclar, Xiang Lorraine Li, Liwei Jiang, Bill Yuchen Lin, Sean Welleck, Peter West, Chandra Bhagavatula, Ronan Le Bras, Jena D. Hwang, Soumya Sanyal, Xiang Ren, Allyson Ettinger, Zaid Harchaoui, and Yejin Choi. 2023. Faith and fate: Limits of transformers on compositionality. In Thirty-seventh Conference on Neural Information Processing Systems. ↗
  12. Lu Wang, and Weizhu Chen. 2022. LoRA: Low-rank adaptation of large language models. In International Conference on Learning Representations.
  13. Jeffrey L. Elman. 1993. Learning and development in neural networks: The importance of starting small. Cognition, 48(1):71–99. https://doi.org/10.1016/0010-0277 (93)90058-4, PubMed: 8403835 ↗
  14. Yuyun Huang and Jinhua Du. 2019. Self-attention enhanced CNNs and collaborative curriculum learning for distantly supervised relation extraction. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 389–398, Hong Kong, China. Association for Computational Linguistics. https://doi.org/10 .18653/v1/D19-1037 ↗
  15. Jerry A. Fodor and Zenon W. Pylyshyn. 1988. Connectionism and cognitive architecture: A critical analysis. Cognition, 28(1-2):3–71. https://doi.org/10.1016/0010-0277 (88)90031-5, PubMed: 2450716 ↗
  16. Robert M. French. 1999. Catastrophic forgetting in connectionist networks. Trends in Cognitive Sciences, 3(4):128–135. https://doi .org/10.1016/S1364-6613(99)01294-2, PubMed: 10322466 ↗
  17. Dieuwke Hupkes, Verna Dankers, Mathijs Mul, and Elia Bruni. 2020. Compositionality decomposed: How do neural networks generalise? Journal of Artificial Intelligence Research, 67:757–795. https://doi.org/10.1613/jair.1.11674 ↗
  18. Xiyan Fu and Anette Frank. 2023. SETI: Systematicity evaluation of textual inference. In Findings of the Association for Computational Linguistics: ACL 2023, pages 4101–4114, Toronto, Canada. Association for Computational Linguistics. https://doi.org/10 .18653/v1/2023.findings-acl.252 ↗
  19. Qi Jia, Yizhu Liu, Haifeng Tang, and Kenny Zhu. 2023. In-sample curriculum learning by sequence completion for natural language generation. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 11937–11950, Toronto, Canada. Association for Computational Linguistics. https://doi.org/10.18653/v1/2023.acl-long.666 ↗
  20. Atticus Geiger, Kyle Richardson, and Christopher Potts. 2020. Neural natural language inference models partially embed theories of lexical entailment and negation. In Proceedings of the Third BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP, pages 163–173, Online. Association for Computational Linguistics. https://doi.org/10.18653/v1/2020.blackboxnlp-1.16 ↗
  21. Nan-Jiang Jiang and Marie-Catherine de Marneffe. 2022. Investigating reasons for disagreement in natural language inference. Transactions of the Association for Computational Linguistics, 10:1357–1374. https://doi.org/10.1162/tacl a 00523 ↗
  22. Yichen Gong, Heng Luo, and Jian Zhang. 2018. Natural language inference over interaction space. In International Conference on Learning Representations. ↗
  23. Xisen Jin, Junyi Du, Arka Sadhu, Ram Nevatia, and Xiang Ren. 2020. Visually grounded continual learning of compositional phrases. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 2018–2029, Online. Association for Computational Linguistics. https://doi.org/10.18653/v1/2020.emnlp-main.158 ↗
  24. Emily Goodwin, Koustuv Sinha, and Timothy J. O’Donnell. 2020. Probing linguistic systematicity. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 1958–1969, Online. Association for Computational Linguistics. https://doi.org/10.18653/v1/2020.acl-main.177 ↗
  25. Aikaterini-Lida Kalouli, Hai Hu, Alexander F. Webb, Lawrence S. Moss, and Valeria De Paiva. 2023. Curing the sick and other Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, NLI maladies. Computational Linguistics, 49(1):199–243. https://doi.org/10.1162/coli a 00465 ization in open domain question answering. In Findings of the Association for Computational Linguistics: NAACL 2022, pages 2014–2029, Seattle, United States. Association for Computational Linguistics. https://doi.org/10 .18653/v1/2022.findings-naacl.155 ↗
  26. Lauri Karttunen. 1971. Implicative verbs. Language, 340–358. https://doi.org/10 .2307/412084 ↗
  27. Wei Liu, Stephen Wan, and Michael Strube. 2024. What causes the failure of explicit to implicit discourse relation recognition? arXiv preprint arXiv:2404.00999. ↗
  28. Daniel Keysers, Nathanael Schärli, Nathan Scales, Hylke Buisman, Daniel Furrer, Sergii Kashubin, Nikola Momchev, Danila Sinopalnikov, Lukasz Stafiniak, Tibor Tihon, Dmitry Tsarkov, Xiao Wang, Marc van Zee, and Olivier Bousquet. 2020. Measuring compositional generalization: A comprehensive method on realistic data. In International Conference on Learning Representations. ↗
  29. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692. ↗
  30. Kai A. Krueger and Peter Dayan. 2009. Flexible shaping: How learning in small steps helps. Cognition, 110(3):380–394. https://doi.org/10.1016/j.cognition.2008.11 .014, PubMed: 19121518 ↗
  31. Andrea Madotto, Zhaojiang Lin, Zhenpeng Zhou, Seungwhan Moon, Paul Crook, Bing Liu, Zhou Yu, Eunjoon Cho, Pascale Fung, and Zhiguang Wang. 2021. Continual learning in task-oriented dialogue systems. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 7452–7467. Online and Punta Cana, Dominican Republic, Association for Computational Linguistics. https://doi.org/10 .18653/v1/2021.emnlp-main.590 ↗
  32. M. Kumar, Benjamin Packer, and Daphne Koller. 2010. Self-paced learning for latent variable models. Advances in Neural Information Processing Systems, 23. find
  33. Philippe Laban, Tobias Schnabel, Paul N. Bennett, and Marti A. Hearst. 2022. SummaC: Re-visiting NLI-based models for inconsistency detection in summarization. Transactions of the Association for Computational Linguistics, 10:163–177. https://doi.org/10 .1162/tacl_a_00453 ↗
  34. Marco Marelli, Stefano Menini, Marco Baroni, Luisa Bentivogli, Raffaella Bernardi, and Roberto Zamparelli. 2014. A SICK cure for the evaluation of compositional distributional semantic models. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14), pages 216–223, Reykjavik, Iceland. European Language Resources Association (ELRA). ↗
  35. Alice Lai, Yonatan Bisk, and Julia Hockenmaier. 2017. Natural language inference from multiple premises. In Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 100–109, Taipei, Taiwan. Asian Federation of Natural Language Processing. ↗
  36. Michael McCloskey and Neal J. Cohen. 1989. Catastrophic interference in connectionist networks: The sequential learning problem. In Psychology of Learning and Motivation, volume 24, pages 109–165. Elsevier. https://doi.org/10.1016 /S0079-7421(08)60536-8 ↗
  37. Brenden M. Lake and Marco Baroni. 2023. Human-like systematic generalization through a meta-learning neural network. Nature, 1–7. ↗
  38. Yuanpeng Li, Liang Zhao, Kenneth Church, and Mohamed Elhoseiny. 2020. Compositional language continual learning. In International Conference on Learning Representations. find
  39. Tom McCoy, Ellie Pavlick, and Tal Linzen. 2019. Right for the wrong reasons: Diagnosing syntactic heuristics in natural language inference. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Linqing Liu, Patrick Lewis, Sebastian Riedel, and Pontus Stenetorp. 2022. Challenges in general-pages 3428–3448, Florence, Italy. Association for Computational Linguistics. https://doi.org/10.18653/v1/P19-1334 ↗
  40. Fei Sha, and Kristina Toutanova. 2022. Evaluating the impact of model scale for compositional generalization in semantic parsing. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 9157–9179, Abu Dhabi, United Arab Emirates, Association for Computational Linguistics. https://doi.org/10 .18653/v1/2022.emnlp-main.624 ↗
  41. Meryem M’hamdi, Xiang Ren, and Jonathan May. 2023. Cross-lingual continual learning. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3908–3943, Toronto, Canada. Association for Computational Linguistics. https://doi.org/10 .18653/v1/2023.acl-long.217 ↗
  42. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. The Journal of Machine Learning Research, 21(1):5485–5551. ↗
  43. Melanie Mitchell, Alessandro B. Palmarini, and Arsenii Kirillovich Moskvichev. 2023. Comparing humans, GPT-4, and GPT-4v on abstraction and reasoning tasks. In AAAI 2024 Workshop on ’’Are Large Language Models Simply Causal Parrots?’’. ↗
  44. Leonardo Ranaldi, Giulia Pucci, and Fabio Massimo Zanzotto. 2023. Modeling easiness for training transformers with curriculum learning. Ruslan Mitkov and Galia Angelova, editors, In Proceedings of the 14th International Conference on Recent Advances in Natural Language Processing, pages 937–948, Varna, Bulgaria. INCOMA Ltd., Shoumen, Bulgaria. https://doi.org/10.26615/978-954-452-092-2_101 ↗
  45. Yixin Nie, Xiang Zhou, and Mohit Bansal. 2020. What can we learn from collective human opinions on natural language inference data? In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 9131–9143, Online. Association for Computational Linguistics. https://doi.org/10.18653/v1/2020.emnlp-main.734 ↗
  46. Mark B. Ring. 1997. Child: A first step towards continual learning. Machine Learning, 28:77–104. https://doi.org/10.1023/A:1007331723572 ↗
  47. Ankur Parikh, Oscar Täckström, Dipanjan Das, and Jakob Uszkoreit. 2016. A decomposable attention model for natural language inference. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 2249–2255, Austin, Texas. Association for Computational Linguistics. https://doi.org/10.18653/v1/D16-1244 ↗
  48. Anthony Robins. 1995. Catastrophic forgetting, rehearsal and pseudorehearsal. Connection Science, 7(2):123–146. https://doi.org/10.1080/09540099550039318 ↗
  49. German I. Parisi, Ronald Kemker, Jose L. Part, Christopher Kanan, and Stefan Wermter. 2019. Continual lifelong learning with neural networks: A review. Neural Networks, 113:54–71. https://doi.org/10.1016/j.neunet.2019.01.012, PubMed: 30780045 ↗
  50. Alexis Ross and Ellie Pavlick. 2019. How well do NLI models capture verb veridicality? In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 2230–2240, Hong Kong, China. Association for Computational Linguistics. https://doi.org/10.18653/v1/D19-1228 ↗
  51. Ellie Pavlick and Tom Kwiatkowski. 2019. Inherent disagreements in human textual inferences. Transactions of the Association for Computational Linguistics, 7:677–694. https://doi.org/10.1162/tacl_a_00293 ↗
  52. Petru Soviany, Radu Tudor Ionescu, Paolo Rota, and Nicu Sebe. 2022. Curriculum learning: A survey. International Journal of Computer Vision, 130(6):1526–1565. https://doi.org/10.1007/s11263-022-01611-x Linlu Qiu, Peter Shaw, Panupong Pasupat, Tianze Shi, Jonathan Herzig, Emily Pitler, Katherine Stasaski and Marti Hearst. 2022. Semantic diversity in dialogue with natural language inference. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 85–98, Seattle, United States. Association for Computational Linguistics. https://doi.org/10 .18653/v1/2022.naacl-main.6 ↗
  53. Felix Hill, Omer Levy, and Samuel Bowman. 2019a. Superglue: A stickier benchmark for general-purpose language understanding systems. Advances in Neural Information Processing Systems, 32.
  54. Hong Wang, Wenhan Xiong, Mo Yu, Xiaoxiao Guo, Shiyu Chang, and William Yang Wang. 2019b. Sentence embedding alignment for lifelong relation extraction. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 796–806, Minneapolis, Minnesota. Association for Computational Linguistics. https://doi.org/10.18653/v1/N19-1086 ↗
  55. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller, Cynthia Gao, Vedanuj Goswami, Naman Goyal, Anthony Hartshorn, Saghar Hosseini, Rui Hou, Hakan Inan, Marcin Kardas, Viktor Kerkez, Madian Khabsa, Isabel Kloumann, Artem Korenev, Punit Singh Koura, Marie-Anne Lachaux, Thibaut Lavril, Jenya Lee, Diana Liskovich, Yinghai Lu, Yuning Mao, Xavier Martinet, Todor Mihaylov, Pushkar Mishra, Igor Molybog, Yixin Nie, Andrew Poulton, Jeremy Reizenstein, Rashi Rungta, Kalyan Saladi, Alan Schelten, Ruan Silva, Eric Michael Smith, Ranjan Subramanian, Xiaoqing Ellen Tan, Binh Tang, Ross Taylor, Adina Williams, Jian Xiang Kuan, Puxin Xu, Zheng Yan, Iliyan Zarov, Yuchen Zhang, Angela Fan, Melanie Kambadur, Sharan Narang, Aurelien Rodriguez, Robert Stojnic, Sergey Edunov, and Thomas Scialom. 2023. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307 .09288.
  56. Sean Welleck, Jason Weston, Arthur Szlam, and Kyunghyun Cho. 2019. Dialogue natural language inference. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3731–3741, Florence, Italy. Association for Computational Linguistics. https://doi.org/10 .18653/v1/P19-1363 ↗
  57. Adina Williams, Nikita Nangia, and Samuel Bowman. 2018. A broad-coverage challenge corpus for sentence understanding through inference. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 1112–1122, New Orleans, Louisiana. Association for Computational Linguistics. https://doi.org/10 .18653/v1/N18-1101 ↗
  58. Tongtong Wu, Massimo Caccia, Zhuang Li, Yuan-Fang Li, Guilin Qi, and Gholamreza Haffari. 2022. Pretrained language model in continual learning: A comparative study. In International Conference on Learning Representations. find
  59. Prasetya Utama, Joshua Bambrick, Nafise Moosavi, and Iryna Gurevych. 2022. Falsesum: Generating document-level NLI examples for recognizing factual inconsistency in summarization. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2763–2776, Seattle, United States. Association for Computational Linguistics. https://doi.org/10 .18653/v1/2022.naacl-main.199 ↗
  60. Benfeng Xu, Licheng Zhang, Zhendong Mao, Quan Wang, Hongtao Xie, and Yongdong Zhang. 2020. Curriculum learning for natural language understanding. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 6095–6104, Online. Association for Computational Linguistics. https://doi.org/10.18653/v1/2020.acl-main.542 Alex Wang, Yada Pruksachatkun, Nikita Nangia, Amanpreet Singh, Julian Michael, Hitomi Yanaka, Koji Mineshima, Daisuke Bekki, and Kentaro Inui. 2020. Do neural models learn systematicity of monotonicity inference in natural language? In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 6105–6117, Online. Association for Computational Linguistics. https://doi.org/10.18653/v1/2020.acl-main.543 Computational Linguistics: Main Volume, pages 920–934, Online. Association for Computational Linguistics. https://doi.org/10.18653/v1/2021.eacl-main.78 ↗
  61. Saizheng Zhang, Emily Dinan, Jack Urbanek, Arthur Szlam, Douwe Kiela, and Jason Weston. 2018. Personalizing dialogue agents: I have a dog, do you have pets too? In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2204–2213, Melbourne, Australia. Association for Computational Linguistics. https://doi.org/10 .18653/v1/P18-1205 ↗
  62. Hitomi Yanaka, Koji Mineshima, and Kentaro Inui. 2021. Exploring transitivity in neural NLI models through veridicality. In Proceedings of the 16th Conference of the European Chapter of the Association for

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/fu-2024-exploring-continual-learning-compositional-generalization/