📄 Identification of Karaka relations in an English sentence

Summary

The paper asks whether karaka relations, the verb-centred dependency labels of Computational Paninian Grammar (CPG), can be assigned automatically to English sentences. The authors say earlier karaka work targeted Indian languages, and they call this the first such effort for English. Their motivation is a natural-language interface to databases (NLIDB) that turns Stanford dependencies into karakas before building semantic frames. A first attempt with a hand-written table translating Stanford labels into karaka tags reached about 52%. They then built a statistical system. It extracts (child, head, karaka) triples from an English Dependency Treebank stored in SSF and aligns them with Stanford parses. Each relation is described by four features: the Levin class of the lemmatized verb, the POS tags of both words and the Stanford label. An SVM classifier trained on these features, with 4-fold cross-validation, reaches roughly 74% accuracy. Inside the NLIDB, the module lowered the syntactic-stage error rate. The work matters because it shows CPG-style relations can be predicted for English from standard parser output.

Key points

• Only six frequent karakas are targeted, which the authors estimate cover about 75% of occurrences: k1 (doer), k2 (object), k3 (instrument), k4 (beneficiary), k5 (source) and k7 (location). k7 is subdivided into time (k7t), place (k7p) and topic (k7).
• The rule-based baseline maps nsubj→k1, dobj→k2, prep_with→k3, prep_to→k4, prep_from→k5, poss→r6 and prep_at→k7, using a preference-based allotment. Its overall performance was 52%.
• The data comes from the English Dependency Treebank in SSF and yields 2557 relations from 1479 sentences (18,345 tokens). Lemmas come from MorphAdorner, and verb classes are based on Beth Levin's classification.
• The classifier is an SVM (LIBSVM, Chang and Lin) tuned by grid search, with c=8.0 and g=0.125 reported as best. The kernels score as follows:
  • linear kernel: 74.42% at c=6.0;
  • polynomial kernel: 74.26% at degree 2;
  • RBF kernel: 74.14% at g=0.125.
• On 213 NLIDB queries, the module reduced the error rate of the syntactic stage by 4.79%.

Interesting ideas

• I1. Because most nouns appeared only once, the authors replaced both words of each relation with POS tags and replaced each verb with a Levin class number keyed on its set of classes, which turns a very sparse lexical problem into a learnable one.
• I2. The fixed table from dependency labels to roles serves as an explicit, inspectable baseline, and a learner trained on the same label plus verb-class features improves on it.
• I3. Training data is built by matching (child, head) pairs between a treebank annotated in a different formalism (SSF/CPG) and an automatic Stanford parse of the same sentence, so the parse's labels are transferred onto the foreign annotation.
• I4. Besides intrinsic accuracy, the module is judged by how much it cuts errors in a downstream application (the NLIDB syntactic stage).
• I5. The location relation is split into time, physical place and topic subtypes, so "when", "where" and "about what" become distinct arguments of the verb.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (frames): Role assignment in prag could back off from the specific verb to its VerbNet class signature when a verb is rare, keeping the decision explainable as "verbs of this class map this label to this role."
• I2 (frames): A UD-relation→role table could stay the transparent default, with learned exceptions conditioned on verb class added only where the table measurably fails.
• I3 (learn): The same pair-alignment trick could harvest silver role labels from treebanks in foreign formalisms by aligning them with our UD parses, with unmatched pairs flagged instead of silently dropped.
• I4 (answer): Changes to role labelling could be scored by their effect on answer accuracy for word problems or fable QA, not only by label accuracy.
• I5 (world): Keeping these three apart when frames are lifted into the world model would feed event ordering (time) and entity placement (place) directly.