📄 Generation that Exploits Corpus-Based Statistical Knowledge

Summary

The paper presents Nitrogen, a general-purpose natural language generator. It addresses a gap between two options: existing "plug-in" generators such as FUF/SURGE, MUMBLE, KPML and RealPro, which need very detailed linguistic input, and templates, which are simple but do not scale. Nitrogen accepts loosely specified meaning structures written in an Abstract Meaning Representation (AMR). These use concepts from the SENSUS ontology, which includes WordNet 1.5. When the input does not settle a linguistic choice, Nitrogen applies a simple lexicon, morphology and a meaning-oriented grammar to produce a word lattice holding many candidate phrasings. A corpus-trained bigram ranker then selects the most fluent path. The algorithm works bottom-up and compositionally, and a recasting mechanism rewrites AMRs to cover non-compositional phenomena such as nested modals. The system has been used in a Japanese–English MT system. Its lattices pack trillions of variants into graphs of hundreds of nodes. The work matters because it moves many hard linguistic decisions out of hand-built knowledge and into corpus statistics, which makes generation robust to incomplete or ambiguous input.

Key points

• The input AMR is a labeled directed graph derived from the PENMAN Sentence Plan Language. Roles can be stated at several levels of depth:
  • shallow semantic roles (:agent, :patient, :sayer),
  • deep syntactic roles (:oblique1–3, which abstract over active and passive voice),
  • plain syntactic roles (:subject, :direct-object).
• The lexicon is a list of 110,000 (word, POS, rank, concept) tuples. It stores no subcategorization, transitivity or countability information; the statistical ranker resolves these softly instead.
• Lexical choice uses the sense ranks: a word is rejected if the target concept is not its primary sense and better words are available. For example, "betray" is preferred over "sell" for |sell<cozen|.
• In the Japanese–English MT system, lattices reach hundreds of nodes and thousands of arcs, encoding trillions of variants. The bigram extractor narrows these to the top N paths.
• Limitations:
  • Bigrams cannot capture long-distance agreement.
  • The system does not generate questions.
  • It does not infer pronouns from coreference.
  • The authors plan to use Penn Treebank statistics to rank tagged lattices and parse forests.

Interesting ideas

• I1. Overgenerate, then rank. Instead of hard-coding defaults (present tense, the first synonym), the generator lists every plausible option in a lattice and lets a language model choose. This pattern suits any pipeline where upstream analysis is unreliable, such as MT from a language that leaves number or tense unmarked.
• I2. Input at variable depth. One formalism accepts roles from shallow syntax up to semantics, so client applications supply whatever level they can produce. The same design also lets researchers test different interlingua abstraction levels without changing the system.
• I3. Keeping ambiguity alive. An OR of concepts is carried through lexical lookup, and words shared by several of the alternatives are preferred. This respects source ambiguity while keeping the output lattice small.
• I4. Rewriting before generation (recasting). The input meaning is transformed and fed back through the generator, so a purely concatenative grammar can still handle structures like nested modals. Normalizing input this way is a cheap alternative to a more powerful grammar formalism.
• I5. Chart-like structure sharing. Generation runs bottom-up, and lattices built for sub-structures are cached and reused. This avoids the repeated work caused by top-down backtracking, much as chart parsing does for analysis.