📄 Hypothesis Transformation and Semantic Variability Rules Used in Recognizing Textual Entailment

Summary

The paper describes a system from Iasi built for the RTE-3 textual entailment challenge. The task is to decide whether a hypothesis can be inferred from a text. Both sentences are parsed into dependency trees with MINIPAR, and named entities are tagged with LingPipe. The method starts from tree edit distance, but instead of scoring raw differences it first rewrites the hypothesis tree so it looks more like the text tree. The rewriting draws on several sources:

• DIRT paraphrase templates for verbs,
• Extended WordNet synonyms for other words,
• an acronym database,
• background knowledge mined semi-automatically from Wikipedia for named entities and numbers.

Each hypothesis node then gets a combined fitness score from four parts: its own mapping, its parent's mapping, edge-label agreement and left/right position. A count of negation and certainty markers can flip the score. A threshold tuned on development data turns the final score into a yes/no decision. The system reached 0.6913 accuracy, with QA pairs easiest and IE pairs hardest. Ablation showed that named-entity checking contributes most. The work shows how far explicit, inspectable tree transformations with lexical resources can go on entailment.

Key points

• Pipeline: MINIPAR dependency parsing and LingPipe NER, then a core module that uses DIRT, WordNet (Extended WordNet), an acronym list and Wikipedia-derived background knowledge. It runs on a peer-to-peer distributed setup to save time.

• Each node's extended local fitness has four components and a maximum of 4. Total fitness is averaged over the hypothesis nodes. A negation value (the share of non-negated verbs) blends that total with its complement, 4 − TF. The decision threshold is 2.06.

• If a named entity in the hypothesis cannot be mapped to the text, even after acronym and background-knowledge lookup, the system answers "no entailment" outright.

• Both submitted runs scored 0.6913 overall. Per task: IE 0.57, IR about 0.69, QA about 0.87, SUM about 0.64.

• Ablation (accuracy without each component):

  Component removed	Accuracy	Drop
  Named entities	0.5758	16.71%
  Negations	0.6763	2.17%
  Background knowledge	—	2.00%
  WordNet	—	1.63%
  Acronyms	—	1.08%
  DIRT	—	0.54%

Interesting ideas

• I1. Rewrite the hypothesis toward the text with DIRT verb templates. When the relation slots cross, as with passive "attacked by" versus active subject/object, swap the argument subtrees, which normalizes voice before any comparison.
• I2. Each node is identified by a triple of its lemma, its parent's lemma and its edge label. This separates repeated words, such as the same name acting as subject in one place and object in another.
• I3. Edge agreement is checked by looking for the hypothesis edge label anywhere on the path between the mapped text nodes. The penalty grows with path length or applies when the label is absent, which tolerates extra intervening structure.
• I4. Negation is treated as parity over markers found below a verb (not, modals and similar). A special rule covers infinitives: words like "allow", "necessary" or "attempt" before "to" keep the action certain, while other contexts weaken it. Hand-built lists of certainty-raising and certainty-lowering words support this.
• I5. Remove one component at a time and measure the accuracy drop. Combine this with a hard veto when a named entity cannot be grounded. The veto proved by far the most valuable signal.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (frames): Frame building could apply the same slot-alignment step so that passive and active paraphrases reach identical who-did-what-to-whom frames, with the template used recorded as the explanation.
• I2 (world): Entity and mention bookkeeping could key on this triple to keep distinct roles of the same lemma apart when filling the world of a text.
• I3 (answer): A reader that matches a question tree against a story tree could use path-based edge matching with graded penalties instead of demanding exact adjacency.
• I4 (frames): Speech-act and modality handling could attach a certainty flag to each event frame using such trigger lists, so later steps know whether an event actually happened.
• I5 (learn): Evaluation could routinely run leave-one-resource-out ablations as a negative control, revealing which knowledge sources really carry the answers.