📄 JU CSE NLP @ SemEval 2017 Task 7: Employing Rules to Detect and Interpret English Puns

Summary

The paper describes a rule-based entry to SemEval-2017 Task 7, which asks systems to decide whether an English sentence contains a pun (Subtask 1) and, if so, which word carries it (Subtask 2). The organizers supplied no training data, so the authors assembled their own collection of 413 pun-bearing sentences, mostly from Project Gutenberg. They used it to estimate simple statistics.

Each sentence is POS-tagged with NLTK and parsed with the Stanford Parser to recover clause structure. Every word then gets a pun score built from several cues:

• a per-POS prior,
• the depth of clause nesting around it,
• its suffix,
• the tags of its neighbours,
• its position in the sentence.

Stop words are fixed at zero. The scores form a "pun vector". If the highest value exceeds 0.25, the sentence is flagged as punning and the top-scoring word is chosen as the pun.

Detection reached F1 of about 0.81 (homographic) and 0.83 (heterographic), largely thanks to high recall. Location was much weaker, at roughly 0.33–0.38. The work shows how far cheap structural heuristics can go on a task that is fundamentally about word senses.

Key points

• The test data for Subtask 1 had 2250 homographic and 1780 heterographic contexts, and each sentence holds at most one pun.
• The POS priors came from the self-collected 413 sentences:
  • noun 0.1538, adjective 0.1111, verb 0.0806, other words 1/(sentence length);
  • 30 of 195 nouns in that set were puns.
• Clauses were sorted into three types by how they nest. Words in Type 3 clauses (the outermost) scored about 0.1 higher.
• Smaller boosts came from several cues:
  • 0.01 for an "-ing"/"-ed" ending;
  • 0.01–0.02 depending on the tag of the previous or next word, with wrap-around;
  • 0.02 for sentence-final position.
• Results:
  • Subtask 1: P 0.7251/0.7369, R 0.9079/0.9402, F1 0.8063/0.8261 (homographic/heterographic);
  • Subtask 2: P = R = F1 0.3348/0.3792, with full coverage.

Interesting ideas

• I1. The system produces a per-word score vector and then makes a single argmax decision with a threshold, so the same structure both detects the phenomenon and locates the word.
• I2. Function words and "be"-verbs are excluded outright with a hard zero before any statistics apply, which shrinks the candidate space cheaply and transparently.
• I3. Clause nesting depth taken from the parse is used as a salience signal, on the observation that words in outer, embedding clauses are likelier to carry the joke.
• I4. Without any supplied training data, the authors built a small corpus by hand and derived human-readable priors from it, rather than training an opaque model.
• I5. Sentence-final position and suffix cues act as cheap positional and morphological evidence for a semantic phenomenon.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (frames): The irony/implicature module could score each token as a candidate trigger and expose the full vector, so a reader can see why one word won and another lost.
• I2 (frames): Pun or wordplay candidates could be restricted to content lemmas from the UD tree (NOUN/ADJ/VERB, excluding copula cop and aux), keeping the rule explicit rather than learned.
• I3 (frames): Depth of ccomp/advcl/conj nesting in the UD tree could serve as an explainable feature for where the punchline or focus of an utterance sits.
• I4 (learn): Small curated sets can yield inspectable count-based priors that seed rule weights. These priors should be checked against negative controls, since the paper's high recall and weak location score hint at overfit thresholds.
• I5 (tokens): Morphology output (suffix, -ing/-ed participles) and token position could be passed upward as explicit features, so later stages can cite them as reasons.