📄 SVO triple based Latent Semantic Analysis for recognising textual entailment

Summary

The paper addresses the RTE-3 task, which asks whether a text T supports the conclusion that a hypothesis H is true. The authors argue that earlier uses of Latent Semantic Analysis for entailment were weakened by treating text as an unordered bag of words, so they make LSA aware of grammatical role. They parsed the Reuters News Corpus with Minipar and extracted subject–verb–object triples through a small AWK and Prolog pipeline. Each word in a triple was stemmed with Snowball and tagged with its role. Co-occurrence counts within triples fed three separate matrices, one each for subjects, verbs and objects, and each matrix was reduced by SVD to 200 dimensions. For each pair, every n-gram of T and H is projected into these spaces. The best match per slot gives three similarity scores, and a hand-tuned weighted threshold makes the decision. The prototype scored 0.5638 accuracy on the development set and 0.5500 on test, modestly above chance. The authors say it beats an earlier LSA entailment system and shows that structured input helps even purely distributional methods.

Key points

• Data: the RTE-3 development set has 800 T–H pairs, half of them positive, spread over the IE, IR, QA and SUM sub-tasks. The sub-task labels were ignored. The only knowledge source was the Reuters corpus (RCV1).
• SVO extraction from Minipar triples covers four clause types:
  • intransitive clauses get an empty object slot;
  • ditransitive clauses become two triples, the second with an empty verb plus the indirect object;
  • predicatives are treated like transitive clauses.
• The matrices were cut to 1000 × 5000 because SVDLIBC (LAS2) never finished on a 5000 × 5000 matrix and failed with "imtqlb failed to converge". The 1000 row stems split into 347 subject, 356 verb and 297 object stems.
• Scoring: average slot similarity is higher for entailing pairs, e.g. subject 0.244 vs 0.196 and object 0.323 vs 0.277. Verb scores are much lower overall. Each slot score is divided by its average, subject and object get weight 3 (σ = ω = 3), and entailment is predicted when q > 14.
• Accuracy was 0.5638 on development and 0.5500 on test. The model is symmetric, so it cannot tell that T entails H from H entailing T.

Interesting ideas

• I1. Build a separate distributional space per grammatical role instead of one shared space, so a word used as an agent is compared only with other agents.
• I2. Compare two texts slot by slot, taking the maximum similarity per role, then divide each slot score by its own corpus average before combining. This stops a naturally low-scoring slot like the verb from being drowned out.
• I3. Encode argument structures as fixed slots with explicit empty fillers: an intransitive clause keeps an empty object, and a ditransitive clause is split into two linked triples.
• I4. When the structural extractor cannot handle a construction, such as clauses embedded inside other clauses, fall back to enumerating candidate spans and let a scoring step choose among them.
• I5. Handle the direction of entailment by checking whether the hypothesis words are more general than the text words, using term frequency as a proxy for specificity.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (world): role-split similarity tables in the compiled knowledge could rank candidate matches between entities and predicates more sharply than a single role-blind embedding.
• I2 (answer): when checking a question frame against text frames, per-role normalised scores would make agent, predicate and patient evidence comparable and easy to explain.
• I3 (frames): a uniform frame shape with explicit "no filler" markers keeps downstream matching simple and makes missing arguments visible rather than silently absent.
• I4 (frames): spans taken from the UD subtree could serve as a logged, explainable fallback filler wherever frame extraction fails on nested clauses.
• I5 (answer): an asymmetric check of general versus specific, backed by WordNet hypernymy or frequency, could stop the reader from accepting a specific claim from a vaguer text.