📄 M3TQA: Massively Multilingual Multitask Table Question Answering

Summary

Research on table question answering with LLMs has focused mostly on English. The few multilingual table benchmarks that exist mainly cover Indo-European or otherwise prominent languages, and they are too small for fine-grained cross-lingual analysis. M3TQA addresses this gap. It starts from 50 real-world tables (41 Chinese, 9 English) that span four structural types and translates them into 97 languages from 12 language families.

Translation runs through a six-stage pipeline built on GPT-4o and DeepSeek:

• structure-preserving initial translation
• filling in missed cells
• refinement at the table level, then at the cell level
• back-translation scored with a length-adaptive BLEU
• a manual check

From these tables the authors build two resources. M3TQA-Bench is a human-annotated test set with four task types. M3TQA-Instruct is an LLM-synthesised training set with no manual labels.

Benchmarking more than 20 models reveals large performance gaps between language families. Fine-tuning on the unannotated synthetic data still yields clear gains, especially for low-resource languages. The work therefore provides both a broad evaluation resource and an inexpensive recipe for extending table reasoning beyond high-resource languages.

Key points

• Translation quality: After discarding roughly 10% of translations with low BLEU, 4,349 multilingual tables remain. Their median back-translation BLEU is 60.19 (minimum 34.06). Linguists' sample checks put cell-level accuracy above 93%.
• M3TQA-Bench: 6,606 professionally annotated QA pairs, made up of 2,916 LLM-generated and 3,690 human-constructed pairs.
  • Tasks: numerical computation, cell extraction, factual verification and open-ended questions.
  • Native-speaker checks estimate annotation accuracy above 95%.
• M3TQA-Instruct: 39,982 synthetic QA pairs. Numerical computation makes up over 70% of them (28,555), a skew the authors list as a limitation.
• Qwen3-8B ablation (average scores):
  • Without thinking: 12.13 base, 28.36 with SFT, 35.06 with GRPO.
  • With thinking: 29.36 base, 39.54 with SFT, 45.17 with GRPO.
• Thinking mode vs. SFT: Thinking mode helps Indo-European, Sino-Tibetan and Uralic languages more. SFT benefits Niger-Congo, Afro-Asiatic and Austronesian languages more. Because all source tables are Chinese or English, some cultural bias remains.