📄 Tralaleros at SemEval-2026 Task 9: Multilingual Polarization Detection with Transformer-based Models

Summary

The paper tackles binary detection of polarized social-media text across 22 typologically diverse languages, using the POLAR data released for SemEval-2026 Task 9, Subtask 1. The authors test four ways of building transformer classifiers, one design question at a time:

• Data rebalancing: undersampling versus NLLB-200 back-translation.
• Hyperparameter search: a random search on mBERT that compares focal loss with cross entropy.
• Encoder scaling: XLM-RoBERTa Large, RemBERT and mDeBERTa-v3, each picked to change a single architectural factor.
• Ensembling: combining the four fine-tuned models.

Three findings stand out. Throwing away majority-class data hurts. XLM-R Large is the best single model on average, yet each encoder has languages where it leads. Ensembling adds almost nothing on top of well-tuned models. The final weighted-average ensemble scored 0.7936 Macro F1 on the test leaderboard and placed third on Hindi and on Telugu.

The work matters because it shows that tricks from English-only setups do not reliably carry over to low-resource languages. It also argues that per-language model routing beats uniform aggregation. The models and an on-device browser extension are publicly released.

Key points

• The training split has 73,681 samples. Overall it is 53.13% polarized, but per-language rates swing from under 10% (Hausa) to over 90% (Khmer). Inputs are truncated to 128 tokens.
• Undersampling removed 12,652 samples and lowered scores in 18 of 22 languages. Back-translation via English with NLLB-200 added 9,542 minority-class samples, about 13% more data.
• The best mBERT run came from 18 random configurations. It used focal loss, frozen embeddings, a backbone LR of 2e−5, a head LR of 1e−4, batch size 16 and three epochs. It reached 0.744 dev Macro F1, against 0.730 for the reference run and 0.381 for the majority baseline.
• XLM-R Large reached 0.7929 dev Macro F1, and the weighted-average ensemble reached 0.7937 (+0.0008). Per-language leaders varied:
  • RemBERT led on Hindi, Telugu, Persian and Urdu.
  • mDeBERTa led on Amharic, Arabic and Turkish.
  • mBERT stayed competitive on Bengali, Burmese, Spanish and Swahili.
• A manual review of 91 errors in Polish, German and English sorted them as follows:
  • implicit political stance: 38%
  • sarcasm or irony: 29%
  • hostility that depends on context or cultural knowledge: 23%
  • subjective borderline cases: 10%

Interesting ideas

• I1. Each alternative encoder was chosen to differ from the baseline along just one dimension (scale/corpus, embedding design, attention/objective), so any gain can be credited to that specific change instead of a tangle of factors.
• I2. A single global ensemble barely helps when the best model differs by subgroup, and the authors suggest a gate that routes each input to its strongest model instead.
• I3. A non-learning majority-class baseline is reported as a floor next to every trained system, which exposes runs that have silently collapsed to one label.
• I4. The error taxonomy (implicit stance, literal reading of sarcasm, missing world knowledge) was built only on languages the annotators speak fluently, so translation could not blur the nuances.
• I5. Freezing pretrained embeddings and giving the new head a much higher learning rate than the backbone proved consistently helpful, whereas an aggressive backbone LR was destructive.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (learn): When comparing parser or tagger variants, change one component per experiment so that evaluation reports can say exactly which design decision bought the improvement.
• I2 (tree): Rather than always merging the perceptron parser and gparse, record which one wins per genre or construction type and route sentences accordingly, keeping the routing decision itself inspectable.
• I3 (learn): Add a trivial predictor as a standing negative control in every evaluation, so a reader or tagger that merely echoes frequent answers is caught immediately.
• I4 (frames): These categories make a ready checklist for the irony and implicature modules, with test cases where the literal content and the intended speech act diverge.
• I5 (tree): When fine-tuning gparse on new treebanks or converted UD dialects, freeze embeddings and decouple head and backbone learning rates to avoid wrecking pretrained representations.