Elamur Research⌕
☰
home / papers / 2003 / Introduction to the Special Issue on the Web as Corpus

Introduction to the Special Issue on the Web as Corpus

Summary

This editorial opens a 2003 special issue of Computational Linguistics devoted to research that draws its linguistic data from the Web. The authors first settle a definitional question. They argue that a corpus should be understood simply as a collection of texts studied as language. Under that definition the Web qualifies. Questions of quality then become a separate, practical matter of whether a given corpus suits a given task.

They place Web-based work in the history of empirical, corpus-driven linguistics, from the Brown corpus through COBUILD to the BNC. Their argument is that Zipfian word distributions leave even 100 million words too sparse for rare words, rare senses and word combinations. They survey early Web-as-corpus research in word sense disambiguation, parallel text mining, language modeling, question answering and ontology population. The introduction also announces Web size estimates, a phrase-translation method, a discussion of representativeness, and the issue's articles.

The editorial matters because it legitimized the noisy but huge Web as a research resource. Its key message is that more data often beats cleverer algorithms.

Key points

  • Hit counts as quick evidence: in a spell-check example, Google returned 67 hits for "speculater" against 82,000 for "speculator".
  • Growth of corpus size:
    • Brown corpus: 1M words, early 1960s.
    • COBUILD: about 8M words by the early 1980s.
    • BNC: 100M words, developed from 1988.
    • The authors stress that corpus size should be thought of in powers of ten.
  • Limits of the BNC: it covers the roughly 10,000 core English words well. Most other words appear fewer than 50 times, which is too few for stable conclusions.
  • Early Web work at ACL (1999–2000):
    • Mihalcea & Moldovan ranked word sense frequencies using search-engine hit counts.
    • Resnik mined bilingual parallel pages from the Web.
    • Jones & Ghani built a language-specific corpus starting from a single document.
    • Fujii & Ishikawa collected definition-like texts.
  • Other resources and findings:
    • The TREC Web track used a frozen sample of about 100GB, mostly from the .gov domain.
    • Banko & Brill scaled training data from 1M to 1B words. Every algorithm they tested kept improving, and the best-performing one changed with data size.

Interesting ideas

  • I1. Keep definitions broad: separate "is X a corpus?" from "is X good for task Y?". This keeps energy on practical evaluation of fitness for a task, which is useful when justifying scraped or uneven Ukrainian data.
  • I2. Use hit counts as cheap statistics: search-engine counts can approximate frequency for spelling choices, sense rankings or attachment decisions. This is a fast sanity check where no large curated corpus exists.
  • I3. Bootstrap a corpus from one seed document: start from a single text in a target language and expand it via the Web. This is attractive for low-resource languages or dialects.
  • I4. Harvest parallel text from bilingual sites: many sites publish the same pages in a local language and a major one. Such sites are a renewable source of parallel data for MT and terminology work.
  • I5. Prefer more data to finer tuning: if performance keeps rising with data size, effort spent collecting data may pay off more than tuning the model. The ranking of methods can also flip at larger scales, so comparisons should be rerun as data grows.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Topics: Word sense disambiguation

Cited by

78 papers of our library cite it (of the 140884 whose bibliographies we read); 3 of them have a page here, 75 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.1162/089120103322711569

@article{kilgarriff2003introduction,
  title = {{Introduction to the Special Issue on the Web as Corpus}},
  author = {Kilgarriff, Adam and Grefenstette, Gregory},
  journal = {Computational Linguistics},
  year = {2003},
  doi = {10.1162/089120103322711569},
  url = {https://doi.org/10.1162/089120103322711569}
}

References

53 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Agirre, Eneko, Olatz Ansa, Eduard Hovy and David Martinez. 2000. Enriching very large ontologies using the WWW. In Proceedings of the Ontology Learning Workshop of the European Conference of AI (ECAI), Berlin. find
  2. Atkins, Sue. 1993. Tools for computer-aided corpus lexicography: The Hector project. Acta Linguistica Hungarica, 41:5–72. find
  3. Atkins, Sue, Jeremy Clear, and Nicholas Ostler. 1992. Corpus design criteria. Literary and Linguistic Computing, 7(1):1–16. find
  4. Baayen, Harald. 2001. Word Frequency Distributions. Kluwer, Dordrecht. find
  5. Baker, Collin F., Charles J. Fillmore, and John B. Lowe. 1998. The Berkeley FrameNet Project. In Proceedings of 11 The idea is developed further in Grefenstette (2001) and in Kilgarriff (2003). find our card
  6. Banko, Michele and Eric Brill. 2001. Scaling to very very large corpora for natural language disambiguation. In Proceedings of the 39th Annual Meeting of the Association for Computational Linguistics and the 10th Conference of the European Chapter of the Association for Computational Linguistics, To… find our card
  7. Beaudouin, Valérie, Serge Fleury, Benoı̂t Habert, Gabriel Illouz, Christian Licoppe, and Marie Pasquier. 2001. Typweb: décrire la toile pour mieux comprendre les parcours. In Colloque International sur les Usages et les Services des Télécommunications (CIUST’01), Paris, June. Available at http:… find
  8. Beesley, Kenneth R. 1988. Language identifier: A computer program for automatic natural-language identification of on-line text. In Language at Crossroads: Proceedings of the 29th Annual Conference of the American Translators Association, pages 47–54, October 12–16. find
  9. Biber, Douglas. 1988. Variation across speech and writing. Cambridge University Press, Cambridge. find
  10. Biber, Douglas. 1993. Using register-diversified corpora for general language studies. Computational Linguistics, 19(2):219–242. find
  11. Briscoe, Ted and John Carroll. 1997. Automatic extraction of subcategorization from corpora. In Proceedings of the Fifth Conference on Applied Natural Language Processing, pages 356–363, Washington, DC, April. find our card
  12. Buitelaar, Paul and Bogdan Sacaleanu. 2001. Ranking and selecting synsets by domain relevance. In Proceedings of the Workshop on WordNet and Other Lexical Resources: Applications, Extensions and Customizations, NAACL, Pittsburgh, June. find
  13. Burnard, Lou. 1995. The BNC Reference Manual. Oxford University Computing Service, Oxford. find
  14. Cavaglia, Gabriela. 2002. Measuring corpus homogeneity using a range of measures for inter-document distance. In Proceedings of the Third International Conference on Language Resources and Evaluation, pages 426–431, Las Palmas de Gran Canaria, Spain, May. find
  15. Church, Kenneth W. and Robert L. Mercer. 1993. Introduction to the special issue on computational linguistics using large corpora. Computational Linguistics, 19(1):1–24. find
  16. Dumais, Susan, Michele Banko, Eric Brill, Jimmy Lin, and Andrew Ng. 2002. Web question answering: Is more always better? In Proceedings of the 25th ACM SIGIR, pages 291–298, Tampere, Finland. find
  17. Fletcher, William. 2002. Facilitating compilation and dissemination of ad-hoc web corpora. In Teaching and Language Corpora 2002. Available at http://miniappolis.com/KWiCFinder/KWiCFinder.html. find
  18. Folch, Helka, Serge Heiden, Benoı̂t Habert, Serge Fleury, Gabriel Illouz, Pierre Lafon, Julien Nioche, and Sophie Prévost. 2000. Typtex: Inductive typological text classification by multivariate statistical analysis for NLP systems tuning/evaluation. In Proceedings of the Second Language Resources… find
  19. Fujii, Atsushi and Tetsuya Ishikawa. 2000. Utilizing the World Wide Web as an encyclopedia: Extracting term descriptions from semi-structured text. In Proceedings of the 38th Meeting of the ACL, pages 488–495, Hong Kong, October. find
  20. Gildea, Daniel. 2001. Corpus variation and parser performance. In Proceedings of the Conference on Empirical Methods in NLP, Pittsburgh, PA. find our card
  21. Greenwood, Mark, Ian Roberts, and Robert Gaizauskas. 2002. University of Sheffield TREC 2002 Q & A system. In E. M. Voorhees and Lori P. Buckland, editors, The Eleventh Text Retrieval Conference (TREC-11), Washington. U.S. Government Printing Office. find
  22. Grefenstette, Gregory. 1995. Comparing two language identification schemes. In Proceedings of the Third International Conference on the Statistical Analysis of Textual Data (JADT’95), pages 263–268, Rome, December 11–13. Available at www.xrce.xerox.com/competencies/content-analysis/publications/Doc… find
  23. Grefenstette, Gregory. 1999. The WWW as a resource for example-based MT tasks. Paper presented at ASLIB “Translating and the Computer” conference, London, October. find
  24. Grefenstette, Gregory. 2001. Very large lexicons. In Walter Daelemans, Khalil Simaan, Jakub Zavrel, and Jorn Veenstra, editors, Computational Linguistics in the Netherlands 2000: Selected Papers from the Eleventh CLIN Meeting, Language and Computers 37. Rodopi, Amsterdam. find
  25. Grefenstette, Gregory and Julien Nioche. 2000. Estimation of english and non-english language use on the WWW. In Proceedings of the RIAO (Recherche d’Informations Assistée par Ordinateur), pages 237–246, Paris. find
  26. Hawking, D., E. Voorhees, N. Craswell, and P. Bailey. 1999. Overview of the TREC8 Web track. In Proceedings of the Eighth Text Retrieval Conference, Gaithersburg, Maryland, November. find
  27. Ipeirotis, Panagiotis G., Luis Gravano, and Mehran Sahami. 2001. Probe, count, and classify: Categorizing hidden Web databases. In Proceedings of the SIGMOD Conference, Santa Barbara, CA. find
  28. Jones, Rosie and Rayid Ghani. 2000. Automatically building a corpus for a minority language from the Web. In Proceedings of the Student Workshop of the 38th Annual Meeting of the Association for Computational Linguistics, Hong Kong, pages 29–36. find
  29. Karlgren, Jussi and Douglass Cutting. 1994. Recognizing text genres with simple metrics using discriminant analysis. In Proceedings of COLING-94, pages 1071–1075, Kyoto, Japan. find
  30. Kessler, Brett, Geoffrey Nunberg, and Hinrich Schütze. 1997. Automatic detection of text genre. In Proceedings of ACL and EACL, pages 39–47, Madrid. find
  31. Kilgarriff, Adam. 2001. Comparing corpora. International Journal of Corpus Linguistics, 6(1):1–37. find
  32. Kilgarriff, Adam. 2003. Linguistic search engine. In Kiril Simov, editor, Shallow Processing of Large Corpora: Workshop Held in Association with Corpus Linguistics 2003, Lancaster, England, March. find
  33. Kilgarriff, Adam and Michael Rundell. 2002. Lexical profiling software and its lexicographical applications—A case study. In Proceedings of EURALEX ’02, Copenhagen, August. find
  34. Kittredge, Richard and John Lehrberger. 1982. Sublanguage: Studies of Language in Restricted Semantic Domains. De Gruyter, Berlin. find
  35. Korhonen, Anna. 2000. Using semantically motivated estimates to help subcategorization acquisition. In Proceedings of the Joint Conference on Empirical Methods in NLP and Very Large Corpora, pages 216–223, Hong Kong, October. find
  36. Lawrence, Steve and C. Lee Giles. 1999. Accessibility of information on the Web. Nature, 400:107–109. find
  37. Manning, Christopher and Hinrich Schütze. 1999. Foundations of Statistical Natural Language Processing. MIT Press, Cambridge. find
  38. McEnery, Tony and Andrew Wilson. 1996. Corpus Linguistics. Edinburgh University Press, Edinburgh. find
  39. Mihalcea, Rada and Dan Moldovan. 1999. A method for word sense disambiguation of unrestricted text. In Proceedings of the 37th Meeting of ACL, pages 152–158, College Park, MD, June. find
  40. Nagao, Makoto. 1984. A framework of a mechanical translation between Japanese and English by analogy principle. In Alick Elithorn and Ranan Banerji, editors, Artificial and Human Intelligence. North-Holland, Edinburgh, pages 173–180. find
  41. Peters, Carol, editor. 2001. Cross-Language Information Retrieval and Evaluation, Workshop of Cross-Language Evaluation Forum (CLEF 2000) Lisbon, Portugal, September 21–22, 2000, Revised Papers. Lecture Notes in Computer Science. Springer-Verlag. find
  42. Resnik, Philip. 1999. Mining the Web for bilingual text. In Proceedings of the 37th Meeting of ACL, pages 527–534, College Park, MD, June. find
  43. Rigau, German, Bernardo Magnini, Eneko Agirre, and John Carroll. 2002. Meaning: A roadmap to knowledge technologies. In Proceedings of COLING Workshop on A Roadmap for Computational Linguistics, Taipei, Taiwan. find
  44. Roland, Douglas, Daniel Jurafsky, Lise Menn, Susanne Gahl, Elizabeth Elder, and Chris Riddoch. 2000. Verb subcategorization frequency differences between business-news and balanced corpora: The role of verb sense. In Proceedings of the Workshop on Comparing Corpora, 38th ACL, Hong Kong, October. find
  45. Sekine, Satshi. 1997. The domain dependence of parsing. In Proceedings of the Fifth Conference on Applied Natural Language Processing, pages 96–102, Washington, DC, April. find
  46. Sinclair, John M., editor. 1987. Looking Up: An Account of the COBUILD Project in Lexical Computing. Collins, London. find
  47. Varantola, Krista. 2000. Translators and disposable corpora. In Proceedings of CULT (Corpus Use and Learning to Translate), Bertinoro, Italy, November. find
  48. Villasenor-Pineda, L., M. Montes y Gómez, M. Pérez-Coutino, and D. Vaufreydaz. 2003. A corpus balancing method for language model construction. In Fourth International Conference on Intelligent Text Processing and Computational Linguistics (CICLing-2003), pages 393–401, Mexico City, February. find
  49. Volk, Martin. 2001. Exploiting the WWW as a corpus to resolve PP attachment ambiguities. In Proceedings of Corpus Linguistics 2001, Lancaster, England. find
  50. Vossen, Piek. 2001. Extending, trimming and fusing WordNet for technical documents. In Proceedings of the NAACL 2001 Workshop on WordNet and Other Lexical Resources, Pittsburgh, June. Available at http://engr.smu.edu/ ∼ rada/mwnw/ papers/WNW-NACL-205.pdf.gz. find
  51. Wilks, Yorick. 2003. On the ownership of text. Computers and the Humanities. Forthcoming. find
  52. Xu, J. L. 2000. Multilingual search on the World Wide Web. In Proceedings of the Hawaii International Conference on System Science (HICSS-33), Maui, Hawaii, January. find
  53. Zheng, Zhiping. 2002. AnswerBus question answering system. In E. M. Voorhees and Lori P. Buckland, editors, Proceedings of HLT Human Language Technology Conference (HLT 2002), San Diego, CA, March 24–27. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/kilgarriff-2003-introduction-special-issue-web-as/