📄 A Unigram Orientation Model for Statistical Machine Translation

Summary

The paper tackles word reordering in phrase-based statistical machine translation, where translation units are blocks, meaning pairs of source and target phrases with no internal structure. The starting point is a monotone block unigram model (Tillmann and Xia, 2003). It is extended so that two neighboring blocks may be swapped, and the swap is guided by a new orientation component. From word-aligned parallel data, the author enumerates blocks and records how often each block follows an adjacent predecessor on its left or on its right. These left/right counts form a simple orientation probability. It is combined log-linearly with a block unigram model and a trigram language model, and decoding uses DP-based beam search. On an Arabic-English task with UN news training data, an LDC devtest set and the DARPA MT 03 blind test set, the orientation model gives a statistically significant BLEU gain over the monotone baseline. Swapping controlled only by the language model performs worse than no reordering at all. The work shows that cheap unigram statistics can make local reordering reliable.

Key points

• Each block has one of three orientations:
  • right, the usual monotone case;
  • left, when the block is swapped with its neighbor;
  • neutral, when there is no adjacent predecessor.
• Neutral orientation is not modeled explicitly. A default model without the orientation term covers it.
• Reordering is limited to swapping neighbor blocks: at most one block is skipped. A block can only be swapped if its left-orientation count passes a threshold.
• Three systems were compared:
  • a monotone baseline;
  • swapping scored only by the trigram LM;
  • swapping plus the orientation model.
• Two interpolation weights were set by exhaustive grid search on the devtest set and reused unchanged on MT 03. Evaluation used case-insensitive BLEU with 4 references and confidence intervals.
• Table 2 lists left vs. right counts for blocks that were swapped. For example, "exhibition"–mErD has 97 left vs. 32 right, and "added"–wADAf has 285 vs. 68.
• The swaps mostly involve nouns and verbs, for example a noun block placed before an adjective block.

Interesting ideas

• I1. Orientation counts without aligning blocks: Instead of computing a Viterbi block alignment, the author enumerates all candidate blocks per sentence pair and checks which of them sit adjacent to each other. This gives cheap reordering statistics, and the same trick could serve any phrase-pair extraction pipeline, including Ukrainian–English data.
• I2. Ignore the neighbor's identity: The model records only whether some block precedes on the left or the right, not which block it is. This avoids sparse pair statistics while still capturing reordering habits, such as nouns tending to move past adjectives.
• I3. Count threshold as a gate: Swapping is allowed only for blocks with enough left-orientation evidence. A frequency gate like this is an easy way to restrict an expensive or risky operation to cases with strong support.
• I4. Weak-baseline ablation: Adding swapping without the orientation term made results worse than no reordering. This shows that extra flexibility needs a matching control signal, and that ablations should test the freedom and its controller separately.
• I5. Fewer, larger units: The unnormalized product of block scores naturally favors translations built from fewer, longer blocks, which tends to help quality. The author argues that full normalization may therefore matter less for this kind of segmentation model.