📄 A Meaning-based English Math Word Problem Solver with Understanding, Reasoning and Explanation

Summary

The paper presents a demonstration system for English arithmetic word problems. It aims to understand the problem text rather than match surface patterns. Stanford CoreNLP produces dependency trees and coreference chains. A linear-kernel SVM then predicts a solution type for each question. A converter turns the parse into first-order-logic facts, and each extracted quantity carries role-tags taken from its syntactic context: its verb, subject, object, head noun and similar. The question becomes a query that calls an inference-engine utility. The engine uses the role-tags to choose the right operands and to discard distractor quantities. A template-based generator then turns the reasoning chain into a readable explanation. On AI2-395 and IL-562 the system scores higher overall than the rule-based ARIS, the statistical KAZB and the mixed UIUC solver. A web interface shows every intermediate output. The paper matters because statistical decisions and symbolic inference are combined in a way that keeps each step inspectable and explainable.

Key points

• The pipeline has five modules: Language Analyzer (CoreNLP), Solution Type Classifier (LIBSVM, linear kernel), Logic Form Converter, Inference Engine and Explanation Generator.
• Each question is assigned one of six categories: the four basic operations (Addition, Subtraction, Multiplication, Division), plus Sum and TVQ (initial, change or final value of a time-variant quantity).
  • Classifier features are verb categories, cue phrases such as "in all" and aggregate patterns over the body.
• Quantities are encoded as facts like quan(q1,#,rose)=36, with auxiliary tags such as verb(q1,pick).
  • qmap links related quantities, for example 100 candies and 5 boxes.
  • The engine offers 9 arithmetic utilities.
• Results:
  • AI2-395 (395 add/subtract problems in subsets MA1, IXL and MA2, 3-fold cross-validation): the system beats UIUC (78.0), ARIS (77.7) and KAZB (64.0) on the micro-average total. With gold solution types, accuracy goes well above 90%.
  • IL-562 (5-fold cross-validation): the system scores 79.5, against 73.9 for UIUC and 73.7 for KAZB.
• Limitations:
  • Only one-step problems are handled.
  • Subtle reference fails, such as linking "customers" to "women" and "men".

Interesting ideas

• I1. Each number is annotated with role-tags from its dependency context, so the number keeps its physical meaning. A question can then select operands by matching tags, such as summing only "pick" quantities and pruning a "drop" one. This makes distractor numbers harmless.
• I2. Parsing to logic happens in two stages: first a direct dump of the dependency tree into predicates, then a separate pass that derives generic math facts and auxiliary tags. This keeps the syntax layer and the domain layer cleanly separated.
• I3. A global classifier first chooses the operation type, and only then is the question compiled into one specific query. The "Gold Solution Type" ablation shows how much error comes from that one decision.
• I4. Explanations are produced by converting the reasoning chain into an explanation tree and filling one template per operation type. This gives fluent output tied exactly to the steps actually used.
• I5. The demo exposes every module's output (parse, coreference, solution type, logic forms, reasoning chain), which makes debugging the pipeline transparent.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (world): Quantity entities in the text-world could store their governing verb, agent and object, so that holdings queries filter by role instead of by position.
• I2 (frames): A frame builder could likewise emit raw UD predicates first and derive role-bearing quantity facts in a later, independently testable pass.
• I3 (answer): The maths reader could log a predicted problem type and be evaluated with an oracle type, so that type errors are measured separately from inference errors.
• I4 (back): Generation from the meaning graph could render solution traces through per-operation templates, which guarantees that the explanation matches the computation.
• I5 (learn): An evaluation dashboard showing per-stage artifacts for failed items would help find which stage needs repair rules.