📄 Fermi at SemEval-2017 Task 7: Detection and Interpretation of Homographic puns in English Language

Summary

This paper tackles two homographic pun subtasks of SemEval-2017 Task 7: deciding whether an English sentence contains a pun, and finding which word carries the pun. For detection, the authors turned an intended unsupervised problem into a supervised one. They hand-labelled a random portion of the data as pun or non-pun and trained a Bi-Directional RNN classifier over 50-dimensional GloVe vectors. For location, they built a rule-based procedure on WordNet. It discards stopwords, punctuation and words with fewer than two senses. It then scores every word pair by the highest embedding similarity across their synsets, takes the best-scoring pair, and returns the member that occurs later in the sentence. The official results were an F1 of 77.65% for detection and 52.15% for location. The location heuristic works reasonably on short sentences but degrades as sentences grow longer. The work shows how far cheap lexical cues, such as polysemy, a semantically related context word and end-of-sentence position, can go on pun processing.

Key points

• Subtask 1 model: an embedding layer feeds a Bi-Directional RNN with dropout 0.5, followed by a fully connected layer with a sigmoid output, trained for 25 epochs.
• Two train/test splits were used for detection:
  • Setting 1 (70/30) reached P 0.9697, R 0.7953, accuracy 0.8360 and F1 0.8738.
  • Setting 2 (30/70) reached P 0.8918, R 0.6876, accuracy 0.7173 and F1 0.7765.
• Subtask 2 used WordNet sense counts to filter candidates and GloVe similarity between synsets (the MAXSIM function) to score pairs. Precision, recall and F1 were all 0.5215, with coverage 1.
• In the raft example, boat–raft is the most similar pair (0.68), so the later word, raft, is chosen as the pun.
• The team did not take part in Subtask 3 (sense interpretation with WordNet sense keys). They suggest it could be extended from their Subtask 2 method.

Interesting ideas

• I1. A pun word must be polysemous, so requiring at least two WordNet senses cheaply removes most candidates before any scoring happens.
• I2. The hidden second meaning of a pun is often signalled by another word in the sentence that fits that meaning (bank→banker, raft→boat), so pairing each candidate with its closest sense neighbour exposes the alternate reading.
• I3. Similarity is computed as the maximum over all synset pairs, not over surface words, so a rare sense can still win when the context supports it.
• I4. A positional prior (the pun tends to come near the end) breaks ties within the chosen pair.
• I5. The authors report that the heuristic weakens as sentences lengthen, because more spurious similar pairs appear; evaluating accuracy by sentence length exposes this.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (frames): The irony/wordplay detector in prag could pre-filter candidates to polysemous content words in the UD tree before checking frames for a double reading.
• I2 (world): When building the text's world, a sense that is strongly supported by an unrelated context entity could be stored as a competing reading, rather than discarded during disambiguation.
• I3 (world): Sense-aware similarity over WordNet synsets could help the ontology layer link entities whose relation holds only under a secondary sense.
• I4 (frames): Punchline position could be one explainable feature when the implicature module decides which word in a joke or fable carries the twist.
• I5 (learn): Mova's evaluations could routinely split scores by sentence length, so that length-sensitive heuristics are caught early.