Elamur Research⌕
☰
home / papers / 2026 / Personalizing News Headlines with Retrieval-Augmented Generation

Personalizing News Headlines with Retrieval-Augmented Generation

Paper · 2026 · Jiajing Wan, Samia Touileb, Lubos Steskal et al.

Summary

The paper asks whether news headlines can be tailored to individual readers by giving an LLM relevant items from that reader's browsing history. The authors build a retrieval-augmented generation (RAG) pipeline on the PENS dataset. PENS contains full article bodies, unlike the LaMP headline task. They restructure its test set into user-specific instances and compare two ways of ranking each user's history against the candidate article: sparse BM25 and a dense retriever. The top-k records are then inserted into the prompt of an instruction-tuned 7–8B model.

Outputs are checked against both the user-written headlines and the original ones, using n-gram overlap, semantic, style-embedding and LLM-judge evaluation. Retrieved history helps most when the records are numerous enough, relevant to the article and topically varied. However, much of the gain comes from copying the general journalistic style found in users' histories rather than their individual taste, so outputs drift toward the original headlines. The authors therefore argue that personalization should be evaluated against two references.

Key points

  • Data: PENS has 113,762 articles in 15 categories. In the test set, 103 native English speakers each rewrote 200 headlines without seeing the originals, and each annotator has a 50-item click history. The authors reorganized this into 20,600 user-specific examples.
  • Generators: Llama-3.1-8B-Instruct, Qwen2.5-7B-Instruct and Granite-3.3-8B-Instruct, all decoded greedily. Llama 3 scored best without user records and became the main model.
  • Retrievers: BM25 (rank_bm25, k1=1.5, b=0.75) versus the dense model bge-base-en-v1.5. Up to k=11 records were used, and BM25-ordered prompts produced better headlines.
  • Evaluation: ROUGE, BLEU, BERTScore, StyleDistance, and LLM-as-a-Judge with Mistral-7B-Instruct-v0.3 and GPT-4o-mini.
  • Annotator rewrites: Original headlines overlap with the rewrites more than any model output does, which suggests users make light edits rather than full redesigns.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of OsloOslo · Norway51 works · 4,155 citations🎓University of BergenBergen · Norway9 works · 583 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksInformation Systems · 1,437 works

🧭 Topics 1Research subjects and languages the work is about.Information retrieval & search · 473 works

Cite

Cite as Jiajing Wan, Samia Touileb, Lubos Steskal and Lilja Øvrelid. 2026. Personalizing News Headlines with Retrieval-Augmented Generation. Proceedings of the Second Workshop on Customizable NLP: Progress and Challenges in Customizing NLP for a Domain, Application, Group, or Individual (CustomNLP4U).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.customnlp4u-1.6

@inproceedings{wan2026personalizing,
  title = {{Personalizing News Headlines with Retrieval-Augmented Generation}},
  author = {Jiajing Wan, Samia Touileb, Lubos Steskal and Lilja Øvrelid},
  booktitle = {Proceedings of the Second Workshop on Customizable NLP: Progress and Challenges in Customizing NLP for a Domain, Application, Group, or Individual (CustomNLP4U)},
  year = {2026},
  doi = {10.18653/v1/2026.customnlp4u-1.6},
  url = {https://doi.org/10.18653/v1/2026.customnlp4u-1.6}
}

References

16 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Mistral AI. 2025. Mistral-7b-instruct-v0.3. https://huggingface.co/mistralai/ Mistral-7B-Instruct-v0.3. Hugging Face model card. ↗
  2. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 311–318, Philadelphia, Pennsylvania, USA. Association for Computational…
  3. Xiang Ao, Xiting Wang, Ling Luo, Ying Qiao, Qing He, and Xing Xie. 2021. PENS: A dataset and generic framework for personalized news headline generation. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural… find
  4. Ajay Patel, Jiacheng Zhu, Justin Qiu, Zachary Horvitz, Marianna Apidianaki, Kathleen McKeown, and Chris Callison-Burch. 2025. Styledistance: Stronger content-independent style embeddings with synthetic parallel examples. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of… find
  5. BAAI. 2023. Baai/bge-base-en-v1.5. https://huggingface.co/BAAI/bge-base-en-v1. 5. Hugging Face model card. ↗
  6. Yilun Qiu, Xiaoyan Zhao, Yang Zhang, Yimeng Bai, Wenjie Wang, Hong Cheng, Fuli Feng, and Tat-Seng Chua. 2025. Measuring what makes you unique: Difference-aware user modeling for enhancing llm personalization. arXiv preprint arXiv:2503.02450. find
  7. Shitao Xiao, Zheng Liu, Peitian Zhang, and Niklas Muennighoff. 2023. C-pack: Packaged resources to advance general chinese embedding. find
  8. Peng Xu, Chien-Sheng Wu, Andrea Madotto, and Pascale Fung. 2019. Clickbait? sensational headline generation with auto-tuned reinforcement learning. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language… find
  9. Han Ren, Xiaona Chang, and Xia Li. 2025. Neural headline generation: A comprehensive survey. Neurocomputing, page 129633. find
  10. Stephen E Robertson, Steve Walker, Susan Jones, Micheline M Hancock-Beaulieu, Mike Gatford, et al. 1995. Okapi at TREC-3. British Library Research and Development Department.
  11. Zhao Yang, Junhong Lian, and Xiang Ao. 2023. Fact-preserved personalized news headline generation. In 2023 IEEE International Conference on Data Mining (ICDM), pages 1493–1498. IEEE. find
  12. Alireza Salemi, Sheshera Mysore, Michael Bendersky, and Hamed Zamani. 2024. LaMP: When large language models meet personalization. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 7370–7392, Bangkok, Thailand. Association for C…
  13. Kui Zhang, Guangquan Lu, Guixian Zhang, Zhi Lei, and Lijuan Wu. 2022. Personalized headline generation with enhanced user interest perception. In International Conference on Artificial Neural Networks, pages 797–809. Springer. find
  14. Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. 2019. Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter. arXiv preprint arXiv:1910.01108. find
  15. Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi. 2019. Bertscore: Evaluating text generation with bert. arXiv preprint arXiv:1904.09675. find
  16. Abigail See, Peter J Liu, and Christopher D Manning. 2017. Get to the point: Summarization with pointer-generator networks. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1073– 1083. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/wan-2026-personalizing-news-headlines-retrieval-augmented/