Elamur Research
☰
home / papers / 2016 / Self-critical Sequence Training for Image Captioning

Self-critical Sequence Training for Image Captioning

Written for the Research library (claude-opus-5-5) from the paper text, in our own words: the license "arXiv — non-exclusive license to distribute" does not allow redistributing the text itself. Original — arxiv.org/pdf/1612.00563

Summary

Image captioning models are usually trained with word-level cross-entropy and teacher forcing. They are then judged with non-differentiable sentence metrics such as CIDEr or BLEU. This creates two problems: exposure bias, and a gap between the training objective and the evaluation metric. Rennie et al. (IBM Watson) address both with self-critical sequence training (SCST), a REINFORCE variant. Instead of learning a separate baseline or critic network, SCST uses the score of the model's own greedy test-time output as the reference reward. A sampled caption is reinforced only when it beats the greedy caption. This lowers gradient variance, needs no extra network, and aligns training with how the model is actually decoded. The authors apply SCST to CNN+LSTM captioners, both plain and attention-based, on MSCOCO, optimising CIDEr directly. Their ensembles set a new state of the art on the official MSCOCO server. The method is simple, cheap and metric-agnostic, and it became a standard fine-tuning step for sequence generation.

Key points

  • Model variants: FC-2K and FC-15K (non-attention) and two spatial-attention LSTMs, Att2in and Att2all, all with 512-dimensional hidden states and embeddings. Training starts with cross-entropy (XE) and continues with SCST.
  • Greedy baseline: the reward of the greedily decoded caption serves as the REINFORCE baseline. This avoids estimating either reward values or normalisation terms.
  • Official MSCOCO server: a 4-model Att2all SCST ensemble raised CIDEr from the previous best of 104.9 to 114.7, with BLEU4 35.2, ROUGE-L 56.3 and METEOR 27.0.
  • Karpathy test split: the 4-model Att2all SCST ensemble scored CIDEr 116.8 with greedy decoding and 117.5 with beam search. SCST outperformed REINFORCE and MIXER-style training of the same architecture.
  • Decoding and robustness: after SCST, greedy decoding performs about as well as beam search. SCST-trained attention models also produced more accurate captions for unusual out-of-context images.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Other

Cited by

255 papers of our library cite it (of the 140673 whose bibliographies we read); 12 of them have a page here, 243 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.1612.00563 · arXiv 1612.00563

@misc{rennie2016self,
  title = {{Self-critical Sequence Training for Image Captioning}},
  author = {Rennie, Steven J. and Marcheret, Etienne and Mroueh, Youssef and Ross, Jarret and Goel, Vaibhava},
  year = {2016},
  doi = {10.48550/arxiv.1612.00563},
  eprint = {1612.00563},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.1612.00563}
}

References

24 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Tsung-Yi Lin, Michael Maire, Serge J. Belongie, Lubomir D. Bourdev, Ross B. Girshick, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C. Lawrence Zitnick. Microsoft COCO: common objects in context. EECV, 2014. 1, 4 find our card
  2. Kyunghyun Cho, Bart van Merrienboer, Çaglar Gülçehre, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. Learning phrase representations using RNN encoder-decoder for statistical machine translation. EMNLP, 2014. 1 find
  3. Oriol Vinyals, Alexander Toshev, Samy Bengio, and Dumitru Erhan. Show and tell: A neural image caption generator. CVPR, 2015. 1, 2 find our card
  4. Andrej Karpathy and Fei-Fei Li. Deep visual-semantic alignments for generating image descriptions. CVPR, 2015. 1, 2 find our card
  5. Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural Comput., 1997. 1, 2 find
  6. Kelvin Xu, Jimmy Ba, Ryan Kiros, Kyunghyun Cho, Aaron C. Courville, Ruslan Salakhutdinov, Richard S. Zemel, and Yoshua Bengio. Show, attend and tell: Neural image caption generation with visual attention. In ICML, 2015. 1, 2, 3 find our card
  7. Marc’Aurelio Ranzato, Sumit Chopra, Michael Auli, and Wojciech Zaremba. Sequence level training with recurrent neural networks. ICLR, 2015. 1, 2, 3, 4, 5, 6, 11 find our card
  8. Samy Bengio, Oriol Vinyals, Navdeep Jaitly, and Noam Shazeer. Scheduled sampling for sequence prediction with recurrent neural networks. In NIPS, 2015. 1, 4, 5, 11 find
  9. Alex Lamb, Anirudh Goyal, Ying Zhang, Saizheng Zhang, Aaron Courville, and Yoshua Bengio. Professor forcing: A new algorithm for training recurrent networks. Neural Information Processing Systems (NIPS) 2016, 2016. 1, 4 find
  10. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: A method for automatic evaluation of machine translation. In ACL, 2002. 2 find our card
  11. Chin-Yew Lin. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out: Proceedings of the ACL-04 workshop, volume 8. Barcelona, Spain, 2004. 2 find
  12. Satanjeev Banerjee and Alon Lavie. Meteor: An automatic metric for mt evaluation with improved correlation with human judgments. In Proceedings of the acl workshop on intrinsic and extrinsic evaluation measures for machine translation and/or summarization, volume 29, pages 65–72, 2005. 2 find our card
  13. Ramakrishna Vedantam, C. Lawrence Zitnick, and Devi Parikh. Cider: Consensus-based image description evaluation. In CVPR, 2015. 2, 11 find our card
  14. Richard S. Sutton and Andrew G. Barto. Reinforcement learning: An introduction. MIT Press, 1998. 2, 3 find
  15. Ronald J. Williams. Simple statistical gradient-following algorithms for connectionist reinforcement learning. In Machine Learning, pages 229–256, 1992. 2, 3 find
  16. John Schulman, Philipp Moritz, Sergey Levine, Michael Jordan, and Pieter Abbeel. High-dimensional continuous control using generalized advantage estimation. arXiv preprint arXiv:1506.02438, 2015. 2 find
  17. Wojciech Zaremba and Ilya Sutskever. Reinforcement learning neural turing machines. Arxiv, 2015. 2, 3 find
  18. Andriy Mnih and Karol Gregor. Neural variational inference and learning in belief networks. arXiv preprint arXiv:1402.0030, 2014. 2 — [arXiv:1402.0030](https://arxiv.org/abs/1402.0030) find
  19. Dzmitry Bahdanau, Philemon Brakel, Kelvin Xu, Anirudh Goyal, Ryan Lowe, Joelle Pineau, Aaron C. Courville, and Yoshua Bengio. An actor-critic algorithm for sequence prediction. Arxiv, 2016. 2 find our card
  20. Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. ICLR, 2015. 3, 5 find
  21. Andrej Karpathy and Fei-Fei Li. Deep visual-semantic alignments for generating image descriptions. In CVPR, 2015. 4, 6, 7 find our card
  22. Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In CVPR, 2016. 4 find
  23. Oriol Vinyals, Alexander Toshev, Samy Bengio, and Dumitru Erhan. Show and tell: Lessons learned from the 2015 MSCOCO image captioning challenge. PAMI, 2016. 7 find
  24. Myung Jinchoi, Antonio Torralba, and Alan S. Willsky. Context models and out-of-context objects. 7, 9, 11, 13, 15, 16 Self-critical Sequence Training for Image Captioning: Supplementary Material find