Elamur Research⌕
☰
home / papers / 2020 / Detecting Foodborne Illness Complaints in Multiple Languages Using English Annotations Only

Detecting Foodborne Illness Complaints in Multiple Languages Using English Annotations Only

Paper · 2020 · Ziyi Liu, Giannis Karamanolakis, Daniel Hsu et al.

Summary

Health departments in several US cities run classifiers that scan online posts, such as Yelp reviews, for reports of food poisoning so that epidemiologists can follow up. These systems work only on English because they were trained on thousands of reviews hand-labeled "Sick" or "Not Sick," and repeating that labeling for each new language is costly. Positive cases are also rare, which makes collecting data even harder. The authors explore cross-lingual text classification that reuses only the existing English labels.

They first show that zero-shot transfer with multilingual BERT (mBERT) handles foodborne-illness nuances less well than it handles tasks like sentiment. To close that gap, they machine-translate the labeled English reviews into target languages and train mBERT on English and translated data together. Experiments cover Yelp reviews in seven languages.

Joint training beats both zero-shot and target-only training. Monolingual BERT on translated data gets the best average F1, but mBERT usually has higher recall and is easier to deploy. Run on unlabeled reviews, the multilingual models catch complaints that English-only pipelines miss.

Key points

  • The task is binary "Sick" / "Not Sick" classification of restaurant reviews, with English labels supplied by epidemiologists. Similar English systems are already used in Chicago, Nevada, New York City and St. Louis.
  • Compared setups:
    • LogReg and monolingual BERT trained on translated target-language data.
    • LogReg* and BERT*, which are trained on English and applied after translating test reviews into English.
    • mBERT in several training configurations: English only (zero-shot), target language only, and joint training.
  • Translating test reviews into English (translate-test) was both weaker and more expensive. It would require translating every incoming review at deployment time.
  • Adding translations into several languages to the training set helped some target languages further. mBERT also showed promise on languages absent from training.
  • The model was applied to unlabeled reviews from the NYC area, the LA area and the Yelp Challenge dataset. Spanish, Chinese and German complaints were flagged "Sick" by mBERT but missed by English LogReg on their machine translations.

Interesting ideas

  • I1. Creating training data by translating existing labeled examples, then training on source and translated data together, gives labels in a new language without new annotation.
  • I2. Translate-test (convert input to the training language at inference) is weaker than translate-train, partly because machine-translated text drifts from the training distribution.
  • I3. The authors note that test sets built by translating English data may overstate quality, since native speakers phrase complaints differently.
  • I4. The translation-garbled Chinese and German examples show how mistranslation alters the event: "not fresh" became "very fresh," and "vomit" became "break." Keyword cues like these mislead a bag-of-words classifier.
  • I5. A single multilingual model avoids pipeline steps such as language detection that can introduce errors. The authors weigh this deployment simplicity against a small accuracy loss.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): Our English labeled frames or silver trees could be projected through translation into extra training views, then checked by our witness parsers before use.
  • I2 (learn): Treat MT-produced or paraphrased input as a separate dialect in evaluation, and include it as a negative-control condition rather than assuming it behaves like native text.
  • I3 (learn): Label any evaluation built from transformed data as such, and keep a small native-text test set to confirm that gains hold beyond the transformation.
  • I4 (frames): Frame-level checks on who experienced what symptom after eating what could expose such meaning flips, instead of trusting surface sentiment words.
  • I5 (tokens): When deciding whether to add a pre-routing stage such as a dialect or register detector, count its error contribution and maintenance cost, not just raw accuracy.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Columbia UniversityNew York · United States254 works · 56,747 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 works

🧭 Topics 2Research subjects and languages the work is about.English · 2,379 worksMultilingual & cross-lingual NLP · 882 works

Cite

Cite as Ziyi Liu, Giannis Karamanolakis, Daniel Hsu and Luis Gravano. 2020. Detecting Foodborne Illness Complaints in Multiple Languages Using English Annotations Only. Proceedings of the 11th International Workshop on Health Text Mining and Information Analysis.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2020.louhi-1.15

@inproceedings{liu2020detecting,
  title = {{Detecting Foodborne Illness Complaints in Multiple Languages Using English Annotations Only}},
  author = {Ziyi Liu, Giannis Karamanolakis, Daniel Hsu and Luis Gravano},
  booktitle = {Proceedings of the 11th International Workshop on Health Text Mining and Information Analysis},
  year = {2020},
  doi = {10.18653/v1/2020.louhi-1.15},
  url = {https://doi.org/10.18653/v1/2020.louhi-1.15}
}

References

14 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Telmo Pires, Eva Schlinger, and Dan Garrette. 2019. How multilingual is multilingual BERT? In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics.
  2. Anna Rogers, Olga Kovaleva, and Anna Rumshisky. 2020. A primer in BERTology: What we know about how BERT works. arXiv preprint arXiv:2002.12327. ↗
  3. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies.
  4. Sebastian Ruder, Ivan Vulić, and Anders Søgaard. 2019. A survey of cross-lingual word embedding models. Journal of Artificial Intelligence Research, 65:569–631. find
  5. Thomas Effland, Anna Lawson, Sharon Balter, Kate-lynn Devinney, Vasudha Reddy, HaeNa Waechter, Luis Gravano, and Daniel Hsu. 2018. Discovering foodborne illness in online restaurant reviews. Journal of the American Medical Informatics Association. find
  6. Adam Sadilek, Henry A Kautz, Lauren DiPrete, Brian Labus, Eric Portman, Jack Teitel, and Vincent Silenzio. 2016. Deploying nEmesis: Preventing foodborne illness by data mining social media. In Proceedings of the AAAI Conference on Artificial Intelligence. find
  7. Stephan Gouws and Anders Søgaard. 2015. Simple task-specific bilingual word embeddings. In Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. find
  8. Shijie Wu and Mark Dredze. 2019. Beto, Bentz, Becas: The surprising cross-lingual effectiveness of BERT. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing.
  9. Jenine K Harris, Leslie Hinyard, Kate Beatty, Jared B Hawkins, Elaine O Nsoesie, Raed Mansour, and John S Brownstein. 2018. Evaluating the implementation of a Twitter-based foodborne illness reporting tool in the City of St. Louis Department of Health. International Journal of Environmental Research and Public Health, 15(5). find
  10. Jenine K Harris, Raed Mansour, Bechara Choucair, Joe Olson, Cory Nissen, and Jay Bhatt. 2014. Health department use of social media to identify foodborne illness-Chicago, Illinois, 2013-2014. Morbidity and Mortality Weekly Report, 63(32):681–685. find
  11. Giannis Karamanolakis, Daniel Hsu, and Luis Gravano. 2019. Weakly supervised attention networks for fine-grained opinion mining and public health. In Proceedings of the 5th Workshop on Noisy User-Generated Text. find
  12. Giannis Karamanolakis, Daniel Hsu, and Luis Gravano. 2020. Cross-lingual text classification with minimal resources by transferring a sparse teacher. In Proceedings of the 2020 Findings of Empirical Methods in Natural Language Processing. find
  13. Kaliyaperumal Karthikeyan, Zihan Wang, Stephen Mayhew, and Dan Roth. 2019. Cross-lingual ability of multilingual BERT: An empirical study. In International Conference on Learning Representations. find
  14. Yu-Hsiang Lin, Chian-Yu Chen, Jean Lee, Zirui Li, Yuyan Zhang, Mengzhou Xia, Shruti Rijhwani, Junxian He, Zhisong Zhang, Xuezhe Ma, et al. 2019. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/liu-2020-detecting-foodborne-illness-complaints-multiple/