Elamur Research⌕
☰
home / papers / 2007 / High-accuracy Annotation and Parsing of CHILDES Transcripts

High-accuracy Annotation and Parsing of CHILDES Transcripts

Summary

The English portion of the CHILDES child language database had only morphological annotation. POS tags alone are not enough to study how a child's syntax develops. The authors added a syntactic layer to these transcripts: grammatical relations (GRs) in the form of labeled dependencies. They revised and extended a previous annotation scheme, in particular adding the labels LOC, ENUM, TOP, SRL and NAME. They also refined the criteria used to distinguish ambiguous parts of speech. They then manually annotated 15 transcript files of a child named Eve. The first 5 thousand words were annotated entirely by hand. After that, a parser trained on the data already completed proposed analyses of new segments, which the annotators checked and corrected, after which the model was retrained. The result is a gold-standard corpus of more than 65 thousand words and a data-driven dependency parser. It accurately parses both child and adult utterances. Both resources are free for research. For psycholinguists, this is a ready-made tool for automatic syntactic analysis adapted to the characteristics of child speech.

Key points

  • The Eve corpus contains 18,863 utterances (10,280 adult and 8,563 child), 65,363 words and 84,226 GRs. Mean utterance length: 5.3 words for adults, 3.6 for the child. The corpus was used in the domain adaptation task of the CoNLL-2007 shared task.
  • The scheme has 37 GR types, and the CHILDES tagset contains 31 POS tags. For the previous version of the scheme, inter-annotator agreement was 96.5%. Each word is described by a triple: index, head index, relation name.
  • The parser trained on Eve was tested on other corpora. On MacWhinney it achieved LAS 88.0 and UAS 91.2. On Seth the result was lower: LAS 84.6. If utterances with filler syllables are removed, LAS rises to 92.0.
  • Typical errors:
    • telegraphic speech: 112 of 5181 errors;
    • confusion of XCOMP and COMP due to an omitted "to": about 150;
    • coordination of prepositional complements: 154;
    • PRED with the verb get: 69.
  • The resources are hosted at childes.psy.cmu.edu. Next, the authors plan to annotate the entire English CHILDES and transfer the approach to the other languages of the database.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cite

Cite as Kenji Sagae, Eric Davis, Alon Lavie, Brian MacWhinney and Shuly Wintner. 2007. High-accuracy Annotation and Parsing of CHILDES Transcripts. Proceedings of the Workshop on Cognitive Aspects of Computational Language Acquisition.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{sagae2007high,
  title = {{High-accuracy Annotation and Parsing of CHILDES Transcripts}},
  author = {Kenji Sagae, Eric Davis, Alon Lavie, Brian MacWhinney and Shuly Wintner},
  booktitle = {Proceedings of the Workshop on Cognitive Aspects of Computational Language Acquisition},
  year = {2007},
  url = {https://aclanthology.org/W07-0604/}
}

References

16 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. A. Berger, S. A. Della Pietra, and V. J. Della Pietra. 1996. A maximum entropy approach to natural language processing. Computational Linguistics, 22(1):39–71. our card
  2. Roger Brown. 1973. A first language: the early stages. George Allen & Unwin Ltd., London. find
  3. Sabine Buchholz and Erwin Marsi. 2006. Conll-x shared task on multilingual dependency parsing. In Proceedings of the Tenth Conference on Computational Natural Language Learning (CoNLL-X), pages 149–164, New York City, June. Association for Computational Linguistics. our card
  4. John Carroll, Edward Briscoe, and Antonio Sanfilippo. 1998. Parser evaluation: a survey and a new proposal. In Proceedings of the 1st International Conference on Language Resources and Evaluation, pages 447–454, Granada, Spain. find
  5. Eugene Charniak. 2000. A maximum-entropy-inspired parser. In Proceedings of the first conference on North American chapter of the Association for Computational Linguistics, pages 132–139, San Francisco, CA, USA. Morgan Kaufmann Publishers Inc. our card
  6. D. Knuth. 1965. On the translation of languages from left to right. Information and Control, 8(6):607–639. find
  7. Brian MacWhinney. 2000. The CHILDES Project: Tools for Analyzing Talk. Lawrence Erlbaum Associates, Mahwah, NJ, third edition. find
  8. Joakim Nivre, Johan Hall, Jens Nilsson, Gulsen Eryigit, and Svetoslav Marinov. 2006. Labeled pseudo-projective dependency parsing with support vector machines. In Proceedings of the Tenth Conference on Computational Natural Language Learning. our card
  9. Joakim Nivre, editor. 2007. CoNLL-XI Shared Task on Multilingual Dependency Parsing, Prague, June. Association for Computational Linguistics. find
  10. Ann M. Peters. 1983. The Units of Language Acquisition. Monographs in Applied Psycholinguistics. Cambridge University Press, New York. find
  11. Ann M. Peters. 1987. The role of immitation in the developing syntax of a blind child. Text, 7:289–311. find
  12. Kenji Sagae and Alon Lavie. 2006. A best-first probabilistic shift-reduce parser. In Proceedings of the COLING/ACL 2006 Main Conference Poster Sessions, pages 691–698, Sydney, Australia, July. Association for Computational Linguistics. find
  13. Kenji Sagae and Jun’ichi Tsujii. 2007. Dependency parsing and domain adaptation with lr models and parser ensembles. In Proceedings of the Eleventh Conference on Computational Natural Language Learning. our card
  14. Kenji Sagae, Alon Lavie, and Brian MacWhinney. 2004. Adding syntactic annotations to transcripts of parent-child dialogs. In Proceedings of the Fourth International Conference on Language Resources and Evaluation (LREC 2004), Lisbon, Portugal. find
  15. Kenji Sagae, Alon Lavie, and Brian MacWhinney. 2005. Automatic measurement of syntactic development in child language. In Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics (ACL’05), pages 197–204, Ann Arbor, Michigan, June. Association for Computational Linguist… find
  16. B. Wilson and Ann M. Peters. 1988. What are you cookin’ on a hot?: A three-year-old blind child’s ‘violation’ of universal constraints on constituent movement. Language, 64:249–273. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/sagae-2007-high-accuracy-annotation-parsing-childes/