A Language Modeling Approach to Information Retrieval

Written for the Research library (claude-opus-5-5) from the paper text, in our own words: the license "author's copy (CIIR UMass) — read only" does not allow redistributing the text itself. Original — ciir.cs.umass.edu/pubfiles/ir-120.pdf

Summary

Classical probabilistic retrieval models need a separate indexing model to estimate term weights, and that indexing step usually rests on parametric assumptions about term distributions. Ponte and Croft avoid this split. They treat each document as a sample from its own language model and rank documents by the probability that this model would produce the query. To estimate each document model, they combine the document's maximum-likelihood term estimates with the average estimate across documents containing the term. A geometric risk function controls the weight of each part. Terms missing from a document fall back to their collection-wide frequency. They compared this query-likelihood ranking with INQUERY tf.idf weighting on TREC ad hoc collections. It gave better precision and recall without any hand-tuned weighting heuristics. The paper matters because it founded the language modeling approach to IR, the basis for later smoothing methods and many modern ranking models.

Key points

• Ranking principle: documents are ordered by query likelihood, the probability that the document's language model generates the query. It replaces explicit relevance-probability modeling.
• Estimation is non-parametric:
  • a document's ML term estimate is blended with the mean estimate over documents containing that term;
  • the blend is weighted by a geometric-distribution risk function;
  • unseen terms get their collection frequency divided by collection size.
• The query is modeled as a set of binary term-occurrence events (a multiple-Bernoulli view). Indexing and retrieval are not treated as separate models.
• Evaluation used TREC ad hoc topics, including topics 202–250, with tf.idf weighting from INQUERY as the baseline. The LM approach beat the baseline on most recall levels and in average precision.
• The authors present the framework as a basis for future work on richer document models and query expansion within the same theory.