可控文本生成技术研究综述

Summary

This Chinese-language survey from Shandong University reviews controllable text generation (CTG). CTG means producing fluent, grammatical text that also meets constraints such as topic, sentiment, style, key entities or persona. The core difficulty is turning those constraints into signals in a latent space that can steer discrete word choice. This gets harder for long texts, which also need varied wording and coherent logic.

The authors organise neural CTG methods into five families:

• fine-tuning pre-trained models
• prompt learning, using either readable instructions or trainable prefix matrices
• conditionally pre-trained models
• attribute-guided decoding
• adversarial (GAN) frameworks

For each family they explain the mechanism and compare strengths, weaknesses and possible improvements in a summary table. Separate sections cover two higher-level goals: diversity of generated text and semantic consistency across sentences in long outputs. The paper also reviews qualitative and quantitative evaluation metrics and labelled datasets. It closes with open problems, including fine-grained sentiment control, hallucination and bias, and discusses future directions informed by cognitive theory.

As a survey, it reports no new experimental results. Its value is a structured map of the field.

Key points

• Published in 中文信息学报, vol. 38, no. 10 (2024), pp. 1–23, with about 157 references. It was funded in part by NSFC grant 62376138.
• The formal setup treats CTG as autoregressive generation conditioned on an input x and an attribute c. GPT, BART, T5, ChatGPT and GPT-4, together with RLHF (PPO and policy-gradient training), are the main backbones discussed.
• Representative methods by family:
  • Fine-tuning: PPLM, CoCon
  • Prompt learning: FLAN, T0, InstructGPT, STRUCTCTG, IP, Prefix-tuning, P-tuning, Multi-Control, Prefix-NLG, DisCup
  • Conditional pre-training: CTRL
  • Attribute-guided decoding: FUDGE, PPCG, CAT-PAW
  • Adversarial: SentiGAN, PT-GAN, SeqGAN, MaskGAN
• GANs struggle because sampling discrete words is not differentiable. The survey describes two workarounds: the Gumbel-softmax approximation and reinforcement-learning rewards supplied by the discriminator.
• Resources cited include:
  • Evaluation: the CTRLEval metric
  • Datasets: DailyDialog, Ubuntu Dialogue Corpus, OpenSubtitles, DBpedia, PubMed 200k RCT, CNewSum and DuReader