Elamur Research⌕
☰
home / papers / 2001 / Error Profiling: Toward a Model of English Acquisition for Deaf Learners

Error Profiling: Toward a Model of English Acquisition for Deaf Learners

Paper · 2001 · Lisa N. Michaud and Kathleen F. McCoy

Summary

ICICLE is a writing tutor for deaf students whose first language is American Sign Language. Its parser uses a broad English grammar extended with mal-rules for typical learner errors, so it often returns several analyses of one ungrammatical sentence. The most plausible analysis depends on how proficient the writer is. The authors want a user model, SLALOM, that records which grammatical features each learner has mastered and that can guess the status of features not yet observed by using a typical order of acquisition.

To estimate that order, they collected writing from 106 deaf individuals and annotated errors with a coding manual whose codes map onto the grammar's mal-rules. They checked inter-coder agreement with Kappa. ESL experts then rated each sample with the TWE scale. The analysis asks which errors are associated with low-, mid- or high-proficiency writers. Errors that mainly appear at lower levels suggest structures acquired earlier.

The initial profiles agree broadly with intuition. Because only errors were counted, data was sparse. The planned fix is to choose the right parse automatically using the human error tags, so that correct usage can also be counted.

Key points

  • The corpus contains writing samples from 106 deaf ASL users, tagged with error codes that correspond to ICICLE's mal-rules.
  • Coding reliability was checked with a Kappa agreement score on a subset of the corpus. Proficiency labels came from ESL instructors using Test of Written English (TWE) ratings.
  • SLALOM labels each grammatical feature as acquired, unacquired, or ZPD (variable performance at the learner's current frontier).
  • An earlier clustering by relative error magnitudes grouped samples in a way that matched their place on the proficiency spectrum. Error profiles then ordered errors by the level that struggles with them most.
  • The population varies widely: about half of deaf 18-year-olds read at or below fourth-grade level, and roughly 10% read above eighth-grade level.

Interesting ideas

  • I1. Use a writer's estimated proficiency as a prior when choosing among competing parses: prefer correct rules for mastered features and mal-rules for unmastered ones.
  • I2. Derive an order of acquisition from cross-sectional data: an error seen only at low proficiency suggests the structure is learned early.
  • I3. Design error codes so they map one-to-one onto grammar mal-rules, which lets human annotations and system output be compared directly.
  • I4. Choose the gold parse by converting each candidate tree's mal-rule constituents into an error-code sequence and string-aligning it with the human tags.
  • I5. The authors note that counting only failures is misleading, and that the right normaliser is how often a structure is attempted.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (tree): When several witness parses disagree on noisy or learner input, a per-source "style profile" could weight which attachment or error reading to keep, and the choice would remain explainable.
  • I2 (learn): The same logic could order curriculum or repair rules by difficulty, using the levels at which Mova's parser still fails on specific constructions.
  • I3 (learn): Self-induced repair rules could carry stable codes, so that LLM-teacher or human labels can be checked against which repair actually fired.
  • I4 (learn): Cheap partial human labels could select the best of several candidate trees, producing silver data without full manual treebanking.
  • I5 (learn): Evaluation of construction-level accuracy should divide errors by occurrences of the construction, not by text length, to avoid misleading sparse statistics.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of DelawareNewark · United States20 works · 1,056 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Arts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksPsychology · 1,552 worksDevelopmental and Educational Psychology · 499 works

🧭 Topics 1Research subjects and languages the work is about.English · 2,263 works

Cite

Cite as Lisa N. Michaud and Kathleen F. McCoy. 2001. Error Profiling: Toward a Model of English Acquisition for Deaf Learners. Proceedings of the 39th Annual Meeting of the Association for Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.3115/1073012.1073063

@inproceedings{michaud2001error,
  title = {{Error Profiling: Toward a Model of English Acquisition for Deaf Learners}},
  author = {Lisa N. Michaud and Kathleen F. McCoy},
  booktitle = {Proceedings of the 39th Annual Meeting of the Association for Computational Linguistics},
  year = {2001},
  doi = {10.3115/1073012.1073063},
  url = {https://doi.org/10.3115/1073012.1073063}
}

References

18 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. N. Bailey, C. Madden, and S. D. Krashen. 1974. Is there a ‘natural sequence’ in adult second language learning? Language Learning, 24(2):235–243. find
  2. C. Baker and D. Cokely. 1980. American Sign Language: A Teacher’s Resource Text on Grammar and Culture. TJ Publishers, Silver Spring, MD. find
  3. Roger Brown and Camille Hanlon. 1970. Derivational complexity and order of acquisition in child speech. In John R. Hayes, editor, Cognition and the Development of Language, chapter 1, pages 11–54. John Wiley & Sons, Inc., New York. find
  4. Jean Carletta. 1996. Assessing agreement on classification tasks: The Kappa statistic. Computational Linguistics, 22(2):249–254, June. ↗
  5. S. P. Corder. 1967. The significance of learners’ errors. International Review of Applied Linguistics, 5(4):161–170, November. ↗
  6. Heidi C. Dulay and Marina K. Burt. 1974. Errors and strategies in child second language acquisition. TESOL Quarterly, 8(2):129–136, June. ↗
  7. Heidi C. Dulay and Marina K. Burt. 1975. Natural sequences in child second language acquisition. Language Learning, 24(1). find
  8. Susan Gass. 1979. Language transfer and universal grammatical relations. Language Learning, 29(2):327–344. find
  9. Diane E. Larsen-Freeman. 1976. An explanation for the morpheme acquisition order of second language learners. Language Learning, 25(1):125– 135, June. ↗
  10. Frank Linton, Brigham Bell, and Charles Bloom. 1996. The student model of the LEAP intelligent tutoring system. In Proceedings of the Fifth International Conference on User Modeling, pages 83– 90, Kailua-Kona, Hawaii, January 2-5. UM96, User Modeling, Inc. find
  11. Kathleen F. McCoy, Christopher A. Pennington, and Linda Z. Suri. 1996. English error correction: A syntactic user model based on principled malrule scoring. In Proceedings of the Fifth International Conference on User Modeling, pages 59– 66, Kailua-Kona, Hawaii, January 2-5. UM96, User Modeling, Inc. find
  12. Lisa N. Michaud and Kathleen F. McCoy. 1999. Modeling user language proficiency in a writing tutor for deaf learners of English. In Mari Broman Olsen, editor, Proceedings of Computer-Mediated Language Assessment and Evaluation in Natural Language Processing, an ACL-IALL Symposium, pages 47–54, College Park, Maryland, June 22. Association for Computational Linguistics.
  13. Lisa N. Michaud and Kathleen F. McCoy. 2000. Supporting intelligent tutoring in CALL by modeling the user’s grammar. In Proceedings of the 13th Annual International Florida Artificial Intelligence Research Symposium, pages 50–54, Orlando, Florida, May 22-24. FLAIRS. find
  14. Lisa N. Michaud, Kathleen F. McCoy, and Christopher A. Pennington. 2000. An intelligent tutoring system for deaf learners of written English. In Proceedings of the Fourth International ACM SIG-CAPH Conference on Assistive Technologies (AS-SETS 2000), Washington, D.C., November 13-15. SIGCAPH. find
  15. Lisa N. Michaud, Kathleen F. McCoy, and Litza A. Stark. 2001. Modeling the acquisition of English: an intelligent CALL approach. In Proceedings of the Eighth International Conference on User Modeling, Sonthofen, Germany, July 13-17. find
  16. Manford Pienemann and Gisela Håkansson. 1999. A unified approach toward the development of Swedish as L2: A processability account. Studies in Second Language Acquisition, 21:383–420. find
  17. David Schneider and Kathleen F. McCoy. 1998. Recognizing syntactic errors in the writing of second language learners. In Proceedings of the Thirty-Sixth Annual Meeting of the Association for Computational Linguistics and the Seventeenth International Conference on Computational Linguistics, volume 2, pages 1198–1204, Université de Montréal, Montréal, Québec, Canada, August 10-14. COLING-ACL, Morgan Kaufmann Publishers. find
  18. M. Strong. 1988. A bilingual approach to the education of young deaf children: ASL and English. In M. Strong, editor, Language Learning and deafness, pages 113–129. Cambridge University Press, Cambridge. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/michaud-2001-error-profiling-model-english-acquisition/