📄 COGEX at RTE 3

Summary

The paper describes LCC's entry, the logic prover COGEX, in the Third PASCAL Recognizing Textual Entailment Challenge (RTE 3). The task is to decide whether a hypothesis H follows from a text T. Both snippets are converted into layered logic forms that carry syntactic, semantic and contextual information. The system then generates axioms from lexical resources, syntax rules and world knowledge, and tries to prove H from T. If a proof fails, H is gradually relaxed, and the final score reflects how much relaxation was needed. A threshold learned on development data turns that score into a YES/NO label.

This year's additions were:

• the eXtended WordNet Knowledge Base, which turns WordNet glosses into semantic relations;
• TARSQI event and temporal annotation;
• explicit logic encoding of events, negation scope, coreference and context;
• lexical-chain axioms built per relation;
• alias and person-name axioms;
• a penalty for named entities in H that cannot be derived from T.

The best run reached 72.25% accuracy on the 800 test pairs. This shows that a fully explainable proof-based pipeline can compete on entailment.

Key points

• Run #1 used TARSQI output (GUTime, Evita, Slinket, GUTenLINK, SputLink) and scored 72.25% test accuracy. Run #2 used LCC's own event and time modules and scored 71.75%. Development accuracy was 76.37% and 74.12% respectively.
• Results varied strongly by subtask in Run #1. QA reached 87.50% and IR 78.00%, but IE (63.50%) and SUM (60.00%) lagged. In IE and SUM, the learned thresholds produced very high recall but low precision.
• The RTE 3 test and dev sets each contain 800 pairs drawn from IE, IR, QA and SUM. About 15% of pairs have long texts (117 in test, 135 in dev), and these average roughly twice as many words.
• Pronoun coreference combines the Hobbs algorithm with RAP (Lappin and Leass). Lexical chains use the coarse-grained WordNet 2.1 sense clusters from SemEval-2007 Task #7 (Navigli, 2006) instead of the top-3 senses.
• The prover used 75 person-name compound axioms on the dev set and 112 on the test set. One example links "Prime Minister Giulio Andreotti" to "Andreotti".

Interesting ideas

• I1. A negation is encoded as an operator over the whole negated event, including its arguments and modifiers, rather than as a flag on the verb, which better matches what the sentence denies.
• I2. A lexical chain is split into one axiom per WordNet relation, each built from a relation-specific template, so the prover can combine them freely instead of compressing a whole chain into one implication.
• I3. Sense candidates come from a coarse cluster around the WSD-chosen sense rather than a fixed top-k, which reduces fragile fine-grained sense distinctions.
• I4. After all inference is done, the system penalises a hypothesis that mentions a named entity with no derivation from the text, a cheap and explainable guard against false positives.
• I5. Person names are decomposed into title, first name and last name, which feeds gender to coreference, stops first names from mapping to wrong WordNet persons, and yields equivalences like full name to surname.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (world): Store negated events in the text's world as negated sub-situations with their roles attached, so a query about a participant or place is not wrongly answered by the bare event.
• I2 (world): Compile ontology links (hypernymy, derivation, verb entailment, agent nouns) as separate small inference rules, so each answer's justification lists the exact relations used.
• I3 (world): Use coarse WordNet sense clusters when linking frame fillers to the ontology, making the knowledge-graph lookup tolerant of tagger sense errors.
• I4 (answer): Before saying "yes", readers could check that every entity in the question is grounded in the text's world, and report the missing entity as the reason.
• I5 (frames): A name-structure step before coreference would let the resolver match "Andreotti" to the full mention and use gender cues, while keeping first names out of ontology linking.