Elamur Research⌕
☰
home / papers / 2019 / A Novel Bi-directional Interrelated Model for Joint Intent Detection and Slot Filling

A Novel Bi-directional Interrelated Model for Joint Intent Detection and Slot Filling

Summary

Spoken language understanding (SLU) requires two linked predictions: a single intent label for an utterance (intent detection, ID) and a tag for each token (slot filling, SF). Earlier joint models connected the tasks only weakly. Some relied on a shared loss. Others, such as the slot-gated model of Goo et al. (2018), passed intent cues to slot tagging but never the reverse.

The authors propose an SF-ID network built on a BLSTM encoder with attention-derived slot and intent context vectors. The SF subnet uses intent information to produce a slot reinforce vector. The ID subnet then pools slot information into an intent reinforce vector. The two subnets exchange these vectors over repeated rounds, which the authors call an iteration mechanism, and an optional CRF layer decodes the slot sequence. Two orderings, SF-First and ID-First, are evaluated on ATIS and Snips.

The model surpasses earlier joint baselines on slot F1, intent accuracy and sentence-level frame accuracy. This suggests that letting information flow both ways between the tasks produces a measurable gain in SLU quality.

Key points

  • On ATIS, the best sentence accuracy is 86.90 (ID-First with CRF), compared with 83.73 for the Slot-Gated baseline, a relative gain of about 3.79%. On Snips, SF-First with CRF reaches 80.57 versus 76.43, a relative gain of about 5.42%.
  • The two modes trade strengths:
    • ID-First gives the top slot F1 (95.80 on ATIS, 92.23 on Snips).
    • SF-First gives the top intent accuracy (97.76 on ATIS, 97.43 on Snips).
  • In the ablation, using only the SF subnet, only the ID subnet, or both subnets without interaction each beats the plain BLSTM. Full interaction between the subnets performs best.
  • Sentence accuracy peaks at three iterations on both datasets and then stays roughly flat with a slight decline. The CRF layer mainly improves slot filling.
  • Setup: BLSTM size 64, Adam optimizer, initial learning rate 0.01 with a decay factor of 0.05. Code is released on GitHub (ZephyrChenzf/SF-ID-Network-For-NLU).

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Cited by

78 papers of our library cite it (of the 140884 whose bibliographies we read); 3 of them have a page here, 75 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/p19-1544

@inproceedings{e2019novel,
  title = {{A Novel Bi-directional Interrelated Model for Joint Intent Detection and Slot Filling}},
  author = {E, Haihong and Niu, Peiqing and Chen, Zhongfu and Song, Meina},
  booktitle = {Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics},
  year = {2019},
  doi = {10.18653/v1/p19-1544},
  url = {https://doi.org/10.18653/v1/p19-1544}
}

References

14 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Yun-Nung Chen, Dilek Hakanni-Tür, Gokhan Tur, Asli Celikyilmaz, Jianfeng Guo, and Li Deng. 2016a. Syntax or semantics? knowledge-guided joint semantic frame parsing. In Spoken Language Technology Workshop (SLT), 2016 IEEE, pages 348–355. IEEE. find
  2. Yun-Nung Chen, Dilek Hakkani-Tür, Gökhan Tür, Jianfeng Gao, and Li Deng. 2016b. End-to-end memory networks with knowledge carryover for multi-turn spoken language understanding. In IN-TERSPEECH, pages 3245–3249. find
  3. Alice Coucke, Alaa Saade, Adrien Ball, Théodore Bluche, Alexandre Caulier, David Leroy, Clément Doumouro, Thibault Gisselbrecht, Francesco Caltagirone, Thibaut Lavril, et al. 2018. Snips voice platform: an embedded spoken language understanding system for private-by-design voice interfaces. arXiv… find our card
  4. Chih-Wen Goo, Guang Gao, Yun-Kai Hsu, Chih-Li Huo, Tsung-Chieh Chen, Keng-Wei Hsu, and Yun-Nung Chen. 2018. Slot-gated modeling for joint slot filling and intent prediction. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human La… find our card
  5. Patrick Haffner, Gokhan Tur, and Jerry H Wright. 2003. Optimizing svms for complex call classification. In Acoustics, Speech, and Signal Processing, 2003. Proceedings.(ICASSP’03). 2003 IEEE International Conference on, volume 1, pages I–I. IEEE. find
  6. Dilek Hakkani-Tür, Gökhan Tür, Asli Celikyilmaz, Yun-Nung Chen, Jianfeng Gao, Li Deng, and Ye-Yi Wang. 2016. Multi-domain joint semantic frame parsing using bi-directional rnn-lstm. In Interspeech, pages 715–719. find
  7. Charles T Hemphill, John J Godfrey, and George R Doddington. 1990. The atis spoken language systems pilot corpus. In Speech and Natural Language: Proceedings of a Workshop Held at Hidden Valley, Pennsylvania, June 24-27, 1990. find
  8. Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980. find
  9. Siwei Lai, Liheng Xu, Kang Liu, and Jun Zhao. 2015. Recurrent convolutional neural networks for text classification. In AAAI, volume 333, pages 2267– 2273. find
  10. Bing Liu and Ian Lane. 2016. Attention-based recurrent neural network models for joint intent detection and slot filling. arXiv preprint arXiv:1609.01454. — [arXiv:1609.01454](https://arxiv.org/abs/1609.01454) find
  11. Christian Raymond and Giuseppe Riccardi. 2007. Generative and discriminative algorithms for spoken language understanding. In Eighth Annual Conference of the International Speech Communication Association. find
  12. Gokhan Tur and Renato De Mori. 2011. Spoken language understanding: Systems for extracting semantic information from speech. John Wiley & Sons. find
  13. Kaisheng Yao, Baolin Peng, Yu Zhang, Dong Yu, Geoffrey Zweig, and Yangyang Shi. 2014. Spoken language understanding using long short-term memory neural networks. In Spoken Language Technology Workshop (SLT), 2014 IEEE, pages 189–194. IEEE. find
  14. Xiaodong Zhang and Houfeng Wang. 2016. A joint model of intent determination and slot filling for spoken language understanding. In IJCAI, pages 2993–2999. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/e-2019-novel-bi-directional-interrelated-model/