Elamur Research⌕
☰
home / papers / 1994 / Training and Scaling Preference Functions for Disambiguation

Training and Scaling Preference Functions for Disambiguation

Paper · 1994 · Hiyan Alshawi and David Carter

Summary

Broad-coverage rule-based analysers generate many competing readings per sentence. To rank them, they combine several preference functions in a weighted sum. Those weights were usually set by experts through trial and error, and the tuning had to be redone for each new domain. Alshawi and Carter learn these "scaling factors" automatically: a least-squares fit supplies starting values, and hill climbing then refines them. The experiments use the Core Language Engine (CLE), which outputs quasi logical form (QLF) analyses, applied to the ATIS corpus. The roughly 650 ATIS trees in the Penn Treebank were too few for reliable results, so the authors semi-automatically built skeletal bracketings for over 4,000 sentences. Each analysis was then scored by how well its constituents overlapped with these trees. The learned weights significantly outperform a laborious hand-tuned set. A second study compares semantic collocation functions over word senses. A newly defined one beats variants based on mutual information and likelihood ratios, lowering disambiguation error from about 30% to about 10%. The study shows that preference weighting can be trained cheaply and that collocation measures should be evaluated together with the other functions.

Key points

  • Method: the weights of a linear combination of preference functions are initialized by squared-error minimization and then improved by hill climbing. This replaces expert hand tuning, which the learned factors significantly outperform.
  • Data: the ATIS-2 training corpus supplied 4615 sentences belonging to classes A and D, none longer than 15 words. The CLE yielded an acceptable constituent structure for 4092 of them (about 89%). Skeletal trees with only two categories, A (argument) and P (predication), were created for these sentences through an interactive tool.
  • Training score: an analysis earns credit for segments it shares with the treebank tree and is penalized for extra segments. The chosen weights were a1=1, a2=10 and a3=0.
  • System setup: about 30 preference functions were tested, and 20 were kept for the spoken language translator (1 collocation-based, 19 others). These include attachment counts and SynRules, a log-probability estimate over syntactic rules.
  • Collocations: the best word-sense collocation function beats both the mutual-information and likelihood-ratio variants and the syntactic-rule-probability function, reducing error from about 30% to about 10%. The method was later reused for reordering N-best speech hypotheses (Rayner et al. 1994).

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🏢AT&TBedminster · United States40 works · 75,466 citations🔬SRI InternationalMenlo Park · United States23 works · 7,208 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

Cite

Cite as Hiyan Alshawi and David Carter. 1994. Training and Scaling Preference Functions for Disambiguation. Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work.

@article{alshawi1994training,
  title = {{Training and Scaling Preference Functions for Disambiguation}},
  author = {Hiyan Alshawi and David Carter},
  journal = {Computational Linguistics},
  year = {1994},
  url = {https://aclanthology.org/J94-4005/}
}

References

21 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Agn~s, M-S., et al. (1993). "Spoken language translator: First year report." SRI International Cambridge Computer Science Research Centre, Report 043. find
  2. Alshawi, H., editor (1992). The Core Language Engine. Cambridge: MIT Press. find
  3. Alshawi, H., and Crouch, R. (1992). "Monotonic semantic interpretation." In Proceedings, 30th Annual Meeting of the Association for Computational Linguistics, Newark, Delaware, 32-39. find
  4. Calzolari, N., and Bindi, R. (1990). "Acquisition of lexical information from a large textual Italian corpus." In Proceedings, 13th International Conference on Computational Linguistics, 3:54-59. find
  5. Chang, J., Luo, Y., and Su, K. (1992). "GPSM: A generalized probabilistic semantic model for ambiguity resolution." In Proceedings, 30th Annual Meeting of the Association for Computational Linguistics, 177-192. find
  6. Church, K. W., and Hanks, P. (1990). "Word association norms, mutual information, and lexicography." Computational Linguistics 16:22-30.
  7. Dagan, I.; Marcus, S.; and Markovitch, S. (1993). "Contextual word similarity and estimation from sparse data." In Proceedings, 31st meeting of the Association for Computational Linguistics, ACL, 164-171. find
  8. Dixon, W. J., and Massey, E J. (1968). Introduction to Statistical Analysis, Third Edition. New York: McGraw-Hill. find
  9. Dunning, T. (1993). "Accurate methods for statistics of surprise and coincidence." Computational Linguistics 19:61-74. find
  10. Gale, W. A., and Church, K. W. (1991). "Identifying word correspondences in parallel texts." In Proceedings, DARPA Speech and Natural Language Workshop, 152-157. Los Altos, California: Morgan Kaufmann.
  11. Hindle, D., and Rooth, M. (1993). "Structural ambiguity and lexical relations." Computational Linguistics 19:103-120.
  12. Hobbs, J. R., and Bear, J. (1990). "Two principles of parse preference." Volume 3, In Proceedings, 13th International Conference on Computational Linguistics, Helsinki, 162-167. find
  13. Marcus, M. P.; Santorini, B.; and Marcinkiewicz, M. A. (1993). "Building a large annotated corpus of English: The Penn treebank." Computational Linguistics 19:313-330.
  14. McCord, M. C. (1989). "A new version of slot grammar." IBM Research Report RC 14506, IBM Thomas J. Watson Research Center, Yorktown Heights, New York. find
  15. McCord, M. C. (1993). "Heuristics for broad-coverage natural language parsing." In Proceedings, ARPA Human Language Technology Workshop, 127-132. Los Altos, California: Morgan Kaufmann. find
  16. Moore, D. S., and McCabe, G. P. (1989). Introduction to the Practice of Statistics. New York/Oxford: Freeman. find
  17. Ostendorf, M.; Kannan, A.; Austin, S.; Kimball, O.; Schwartz, R.; and Rohlicek, J. R. (1991). "Integration of diverse recognition methodologies through reevaluation of N-best sentence hypotheses." In Proceedings, DARPA Speech and Natural Language Workshop, 83-87. Los Altos, California: Morgan Kaufma… find
  18. Pereira, F., Tishby, N., and Lee, L. (1993). "Distributional clustering of English words." In Proceedings, 31st meeting of the Association for Computational Linguistics, ACL, 183-190.
  19. Rayner, M.; Carter, D. M.; Digalakis, V.; and Price, P. (1994). "Combining knowledge sources to reorder N-best speech hypothesis lists." In Proceedings, ARPA Human Language Technology Workshop, in press. find
  20. Resnik, P., and Hearst, M. A. (1993). "Structural ambiguity and conceptual relations." In Proceedings, Workshop on Very Large Corpora, ACL, 58-64. find
  21. Sekine, S.; Carroll, J. J.; Ananiadou, S.; and Tsujii, J. (1992). "Automatic learning for semantic collocation." In Proceedings, Third Conference on Applied Natural Language Processing, ACL, 104-110. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/alshawi-1994-training-scaling-preference-functions-disambiguation/