📄 Sentence Alignment for Monolingual Comparable Corpora

Summary

The paper tackles sentence alignment between monolingual comparable texts. These are documents in one language that cover overlapping content but differ heavily in what they include, how they order it, and how they phrase it. The goal is to automatically collect sentence pairs that can serve as training data for learning text-to-text rewriting rules, for example for simplification or summarization.

The method keeps the sentence similarity measure deliberately simple (cosine word overlap) and adds context at the paragraph level in two stages:

• Macro alignment: paragraphs in each collection are clustered into domain topics without supervision, and rules for mapping topics between the two collections are learned from manually aligned text pairs.
• Micro alignment: sentences are aligned locally inside each mapped paragraph pair.

The evaluation uses encyclopedia-style texts in elementary and comprehensive versions. The approach beats the Cosine baseline, Decomposition (an HMM) and SimFinder, and it recovers matches even when two sentences share few words. Even a fully unsupervised variant that uses only paragraph-level lexical similarity outperforms systems built on richer sentence-similarity functions.

Key points

• In one example text pair, only 35 of the 31 × 270 possible sentence pairs are true matches, and their order varies widely between the texts. This shows why MT-style monotonic alignment assumptions break down for comparable corpora.
• Topics are induced by hierarchical complete-link clustering of paragraphs, with cosine similarity over content words. Proper names, dates and numbers are replaced by generic tags before clustering.
• Adding the context-based search on top of Cosine yields a 19% performance improvement at 55.8% Recall.
• Ablations at 55.8% Recall:
  • Removing local alignment drops Precision to 73.3%, which is 3% below the full method.
  • Replacing automatic clusters with manually derived "ideal" clusters (8 for the elementary version, 11 for the comprehensive one) adds only 4% Precision.
• For sentence pairs with low lexical overlap (0–40%), the full method reaches 50% Precision, compared with 34% for Decomposition and 23% for Cosine.

Interesting ideas

• I1. Lean on context, not a heavy similarity function. Pairing a cheap lexical score with a search over context can outperform a sophisticated sentence-level scorer. This is a useful default when building alignment or paraphrase resources for languages with few semantic tools.
• I2. Strip text-specific details before clustering. Masking names, dates and numbers groups paragraphs by the kind of information they carry, such as "climate" or "symptoms", rather than by the specific entity described. The same trick could help induce document schemas in other genres, like news or trade reports.
• I3. Align coarse-to-fine. First decide which paragraphs could plausibly correspond, then align sentences only inside those pairs. This sharply shrinks the search space and suppresses spurious matches in noisy, reordered document pairs.
• I4. Let neighboring pairs support weak matches. Local alignment lets a lexically weak sentence pair be accepted when the pairs around it match well. This idea could transfer to aligning loosely related bilingual or multi-version documents.
• I5. Test sensitivity to cluster quality with ideal clusters. Using existing section headings to build "ideal" clusters gives a cheap upper bound on how much better clustering would help. The same evaluation trick works for any pipeline that depends on an induced intermediate structure.