📄 Neural Cross-Lingual Named Entity Recognition with Minimal Resources

Summary

The paper tackles named entity recognition for languages with no labeled data, transferring knowledge from English annotations. It addresses two obstacles: mapping vocabulary between languages cheaply, and handling differences in word order. For vocabulary, the authors align separately trained monolingual embeddings into a shared space using a small seed dictionary (orthogonal Procrustes plus iterative self-learning). Each English training word is then replaced by its nearest target-language neighbour under the CSLS metric, and the entity tags are carried over. Because the synthetic training data contains real target-language words, the tagger can still use character-level features. For word order, they add a self-attention layer to a Bi-LSTM-CRF tagger so it can gather context more flexibly. On CoNLL 2002/2003 data, the approach gives state-of-the-art cross-lingual results for Spanish and Dutch and competitive results for German, without Wikipedia or parallel corpora. Uyghur experiments show it still helps in a truly low-resource setting, although poor embeddings limit performance. The work lowers the resource requirements for building NER in under-served languages.

Key points

• Requirements are minimal: source-language labels, monolingual corpora in both languages, and a small dictionary, which can be pre-existing or induced without supervision. Embedding alignment uses k = 3 self-learning refinement steps.
• In an internal comparison of embedding-use strategies, translating the training data outperformed both the shared-space and word-replacement variants. F1 scores were 69.21 for Spanish, 69.39 for Dutch, and 53.94 for German, where the shared-space variant reached only 43.73.
• On Uyghur (199 DARPA LORELEI "unsequestered set" documents), the method used just a 5K dictionary and fastText/GloVe embeddings. It scored 26.38 F1 with self-attention, compared with 27.20 for Mayhew et al. (2017) using English data, Wikipedia and a 100K dictionary.
• The best Uyghur result, 32.09 F1, came from combining Mayhew et al.'s dictionary with embedding-based translation of uncovered words.
• Fully unsupervised alignment (adversarial training or identical strings) failed for Uyghur, with F1 around 10. The authors attribute this to low-quality embeddings, the distance between the languages, and the Arabic script.