FeTaQA: Free-form Table Question Answering

Written for the Research library (claude-opus-5-5) from the paper text. Original — aclanthology.org/2022.tacl-1.3.pdf

Summary

Most table question answering benchmarks expect a short entity or value as the answer. Real users, however, often ask questions that require combining several facts from a table into a full sentence. To address this gap, the authors built FeTaQA, a dataset of 10,330 Wikipedia table questions with free-form answers. They did not have annotators write both questions and answers. Instead, they took table-grounded sentences from ToTTo, kept tables of moderate size whose highlighted cells span multiple rows or columns, and asked annotators to write questions that those sentences answer. The highlighted cells also serve as weak supervision.

Two baselines were tested:

a pipeline in which TAPAS selects cells and a separate generator writes the answer;
an end-to-end T5 model that reads a linearized table.

The end-to-end model clearly outperformed the pipeline on automatic metrics and in human evaluation, but it still fell well short of the reference answers. FeTaQA is the first generative QA benchmark over tables, and it tests whether a system can retrieve, reason over, and integrate structured data.

Key points

Dataset composition: FeTaQA has 10,330 instances. Internal experts produced 3,039 (30%) and Amazon Mechanical Turk workers produced 7,291 (70%). Reviewers rejected 12% of crowd annotations and edited only 16.7% of those they approved.
Answer length: Answers average 18.9 words, compared with 1.7 in WikiTableQuestions and 2.1 in HybridQA.
Human evaluation (share of answers rated 4–5):

Answer source Correct Adequate Faithful

End-to-End T5 54.8% 48.4% 50.4%

Pipeline 25.4% 8.4% 23.6%

References 92.4% 90.6% 95.6%
Pipeline weakness: Gold annotations highlight a median of six cells, but the TAPAS parser predicts only about two cells, or about three after fine-tuning. Fine-tuning raises the pipeline by nearly 2 BLEU. With gold cells, the pipeline beats End-to-End.
Availability: The data and code are released at github.com/Yale-LILY/FeTaQA.

Answer source	Correct	Adequate	Faithful
End-to-End T5	54.8%	48.4%	50.4%
Pipeline	25.4%	8.4%	23.6%
References	92.4%	90.6%	95.6%

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Cited by

123 papers of our library cite it (of the 140673 whose bibliographies we read); 6 of them have a page here, 117 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.1162/tacl_a_00446

@article{nan2022fetaqa,
  title = {{FeTaQA: Free-form Table Question Answering}},
  author = {Nan, Linyong and Hsieh, Chiachun and Mao, Ziming and Lin, Xi Victoria and Verma, Neha and Zhang, Rui and Kryściński, Wojciech and Schoelkopf, Hailey and Kong, Riley and Tang, Xiangru and Mutuma, Mutethia and Rosand, Ben and Trindade, Isabel and Bandaru, Renusree and Cunningham, Jacob and Xiong, Caiming and Radev, Dragomir},
  journal = {Transactions of the Association for Computational Linguistics},
  year = {2022},
  doi = {10.1162/tacl\_a\_00446},
  url = {https://doi.org/10.1162/tacl_a_00446}
}

References

1 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

Sumit Asthana and Aaron Halfaker. 2018. With few eyes, all hoaxes are deep. In Proceedings of the ACM on Human Computer Interaction 2(CSCW). https://doi.org/10 .1145/3274290 find