Elamur Research⌕
☰
home / papers / 2018 / Adversarial Deep Averaging Networks for Cross-Lingual Sentiment Classification

Adversarial Deep Averaging Networks for Cross-Lingual Sentiment Classification

Summary

The paper addresses sentiment classification in languages with little or no annotated data. It transfers knowledge from labeled English (SOURCE) texts to a TARGET language that has only unlabeled text. The authors propose ADAN, which feeds a Deep Averaging Network (DAN) feature extractor into two branches: a sentiment classifier trained on English labels and an adversarial language discriminator. The extractor is pushed to produce features that predict sentiment well but do not reveal the input's language. Instead of the usual Gradient Reversal Layer, ADAN reduces the Wasserstein distance between the English and target feature distributions, which makes training smoother. The only bilingual resource is a set of bilingual word embeddings (BWEs), which need far less parallel data than a full MT system.

On Chinese and Arabic, ADAN significantly beats baselines and state-of-the-art CLSC systems, including an MT-based approach and Xu and Yang's (2017) distillation method. It also stays ahead when some target labels are available. This makes ADAN a cheap option for building sentiment tools in low-resource languages.

Key points

  • The setup uses English as SOURCE and Chinese and Arabic as TARGETs. The comparison systems are a DAN trained only on English with BWEs, the mSDA domain-adaptation method (Chen et al., 2012), an LR+MT baseline, and cross-lingual distillation (Xu and Yang, 2017).
  • With randomly initialized embeddings and no bitext at all, ADAN still beats the BWE-based DAN and mSDA baselines and matches LR+MT. Results are also robust to which BWEs are chosen.
  • A grid search over k ∈ {1, 2, 4, 8, 16} and λ from 0.00625 to 0.8 shows that the Wasserstein version is more accurate and more stable than ADAN-GRL on the Chinese dev set.
  • Implementation details:
    • F has 3 ReLU layers, and P and Q have 2 layers each, all with 900 hidden units.
    • Both optimizers are Adam with a learning rate of 0.0005.
    • Q's weights are clipped to [−0.01, 0.01].
    • Training runs for 30 epochs with early stopping, in PyTorch.
  • The code is public at github.com/ccsasuke/adan, and the paper is CC-BY 4.0 (TACL vol. 6, 2018).

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Cited by

100 papers of our library cite it (of the 140884 whose bibliographies we read); 11 of them have a page here, 89 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.1162/tacl_a_00039

@article{chen2018adversarial,
  title = {{Adversarial Deep Averaging Networks for Cross-Lingual Sentiment Classification}},
  author = {Chen, Xilun and Sun, Yu and Athiwaratkun, Ben and Cardie, Claire and Weinberger, Kilian},
  journal = {Transactions of the Association for Computational Linguistics},
  year = {2018},
  doi = {10.1162/tacl\_a\_00039},
  url = {https://doi.org/10.1162/tacl_a_00039}
}

References

2 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Martin Arjovsky, Soumith Chintala, and Léon Bottou. 2017. Wasserstein generative adversarial networks. In Proceedings of the 34th International Conference on Machine Learning, volume 70, pages 214–223, Sydney, Australia. find
  2. Conclusion and Future Work Carmen Banea, Rada Mihalcea, and Janyce Wiebe. 2010. Multilingual subjectivity: Are more languages better? In Proceedings of the 23rd International Conference on Computational Linguistics (COLING 2010), pages 28–36, Beijing, China. find

This page in Markdown → https://md.research.elamur.ai/free/en/papers/chen-2018-adversarial-deep-averaging-networks-cross/