📄 Hafez: an Interactive Poetry Generation System

Summary

Hafez is a demo system that writes English sonnet-style poems on any topic a user supplies. It targets three weaknesses of earlier poetry generators: poem quality is hard to evaluate, users cannot steer a draft beyond changing the random seed, and generation is slow. The pipeline builds on the authors' earlier work. A word2vec model and a rhyme-type dictionary choose topic-related rhyme pairs. A Finite State Acceptor (FSA) then enforces meter and line-final rhymes, and an RNN decoder fills in the lines under the FSA's guidance. The new version adds eight weighted feature functions to the beam-search score, which are exposed as sliders for style control. A web interface lets users rate poems from 1 to 5 stars, and these logs are used to learn a better default style configuration. GPU porting and vocabulary pruning cut generation time from about 20 seconds to 2. In Mechanical Turk experiments, most users found an adjusted poem they preferred to the default one. This shows that interactive control plus crowd feedback can improve creative generation.

Key points

• Style features. Eight features are added to the beam score:

  • encouraged or banned words, with fixed weights of +5 and −5;
  • curse words;
  • repetition;
  • alliteration;
  • word length;
  • topical words;
  • sentiment, based on SentiWordNet;
  • concreteness, based on Brysbaert et al. (2014) norms.

  The seven tunable weights form the "style configuration."

• Speed. A four-line poem takes about 2 seconds, versus roughly 20 seconds for the earlier CPU system. The gains come from:

  • preloading the 1.8M-entry word2vec vocabulary, which saves 3–4 seconds;
  • precomputing rhyme types;
  • restricting candidate rhyme words to the RNN vocabulary;
  • running scoring and top-k selection on the GPU.

• Mechanical Turk study (62 HITs).

  • In 71% of HITs, at least one adjusted poem was rated higher than the default poem.
  • Only 10% of reported best poems came from the default settings.
  • Best poems averaged +1.4 stars over the default.
  • A separate check with 18 users compared default and best poems, to control for creators being biased toward their own edits.

• Other uses. The system also runs as an Amazon Alexa Skill ("Custom Poetry"), which recites poems aloud. It was also used in a 15-minute poetry contest with 7 participants.

Interesting ideas

• I1. Hard constraints in an automaton, fluency in a neural model. An FSA guarantees form (meter, rhyme placement), while the RNN supplies fluent wording within those limits. This split could suit any generation task with strict structural rules, such as templates, metrical verse in other languages, or slot-constrained text.
• I2. Style as additive scoring terms. Each controllable attribute is a simple per-word feature with a weight added to the beam-search score. This gives cheap, interpretable, user-facing knobs without retraining the model, and it carries over to any decoder.
• I3. "Same rhyme words, new style" comparisons. Users regenerate while holding the rhyme set fixed, and each rating is recorded relative to the default poem. Every slider change therefore becomes a paired comparison, which reduces noise in subjective ratings.
• I4. Per-weight quadratic fits for tuning. A quadratic curve is fitted between each weight's offset from the default and the change in star rating, and the peak of that curve becomes the new default. This lightweight way of tuning hyperparameters from crowd feedback could work wherever only noisy human scores are available.
• I5. Speed as a prerequisite for evaluation. The authors treat speed not just as a convenience but as what makes large-scale feedback collection possible. One example is replacing a per-beam CPU heap with a single global GPU sort, which is asymptotically worse but faster in practice. The broader lesson is to optimize interactive systems so that human evaluation can scale.