A Multi-View Media Profiling Suite: Resources, Evaluation, and Analysis
Paper · 2026 · Muhammad Arslan Manzoor, Dilshod Azizov, Daniil Orel et al.
Summary
Automatically judging news outlets for political bias and factuality is a scalable alternative to checking individual claims or articles. Progress has been held back by fragmented resources, small label sets, and evaluations that use only one graph view on one benchmark. The authors build a profiling suite around MBFC-2025, a new source-level label set of about 2,600 outlets. Each outlet is labelled by Media Bias/Fact Check on 5-point scales for bias and factuality. For both MBFC-2025 and the older ACL-2020 set, they assemble five representations of each outlet:
- three graph views: Alexa audience-overlap graphs, crawled hyperlink graphs, and graphs built from GPT-3.5 suggestions of similar sites;
- two text views: articles and Wikipedia descriptions.
Graph views are encoded with GNNs and text views with PLMs. The authors then compare fusion strategies, including a contextual-bandit RL weighting scheme. The study reports state-of-the-art results on ACL-2020 and sets baselines for MBFC-2025. The Alexa graph proves the most useful view, and RL fusion mainly helps political bias detection. This matters because it shows which signals remain reliable when modalities are sparse or missing.
Key points
- Label sets:
- The ACL-2020 sample used has 475 outlets on 3-point scales.
- MBFC-2025 has 2,550 outlets on 5-point scales, adding left-center, right-center, very high, and very low.
- In MBFC-2025, bias labels are dominated by right-center (759) and center (705), and factuality labels by high (1,347) and mixed (970).
- Graph sizes:
- The Alexa graph (level 3) has 67,350 nodes and 100,261 edges.
- The MBFC-2025 hyperlink graph reaches 528,214 nodes and about 2M edges.
- The MBFC-2025 LLM-graph has 80,385 nodes.
- Graph construction:
- Hyperlinks were collected with newspaper3k, using up to 50 articles per site.
- The LLM-graph was built by asking gpt-3.5-turbo-0125 for five similar sites per outlet, then merging duplicate nodes.
- Main findings:
- Alexa is the strongest view, thanks to its node features.
- Hyperlink graphs add complementary signal.
- The LLM-graph is weakest, because it has only dummy node features.
- Scores on the finer-grained MBFC-2025 are lower than on ACL-2020.
- Limitations:
- The label categories are U.S.-centric.
- Some websites could not be parsed.
- Only code, URLs, and labels are released, not the articles or graphs themselves.
Institutions
Fields, topics and concepts
📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksSocial Sciences · 1,772 worksSociology and Political Science · 577 works
Cite
Cite as Muhammad Arslan Manzoor, Dilshod Azizov, Daniil Orel, Umer Siddique, Zain Muhammad Mujahid, Yufang Hou and Preslav Nakov. 2026. A Multi-View Media Profiling Suite: Resources, Evaluation, and Analysis. Findings of the Association for Computational Linguistics: ACL 2026.
@inproceedings{manzoor2026multi,
title = {{A Multi-View Media Profiling Suite: Resources, Evaluation, and Analysis}},
author = {Muhammad Arslan Manzoor, Dilshod Azizov, Daniil Orel, Umer Siddique, Zain Muhammad Mujahid, Yufang Hou and Preslav Nakov},
booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
year = {2026},
doi = {10.18653/v1/2026.findings-acl.1264},
url = {https://doi.org/10.18653/v1/2026.findings-acl.1264}
}References
- Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. GPT-4 technical report. arXiv preprint arXiv:2303.08774.
- Sadia Afroz, Michael Brennan, and Rachel Greenstadt. 2012. Detecting hoaxes, frauds, and deception in writing style online. In Proc. of the 2012 IEEE Symposium on Security and Privacy, page 461–475, San Francisco, CA, USA. IEEE Computer Society. find
- Dilshod Azizov, Zain Muhammad Mujahid, Hilal AlQuabeh, Preslav Nakov, and Shangsong Liang. 2024. Safari: Cross-lingual bias and factuality detection in news media and news articles. In Findings of the Assoc. for Comp. find
- Linguistics: EMNLP 2024, pages 12217–12231, Miami, Florida, United States. find
- Dilshod Azizov, Preslav Nakov, and Shangsong Liang. 2023. Frank at checkthat!-2023: Detecting the political bias of news articles and news media. In Working Notes of the Conference and Labs of the Evaluation Forum (CLEF 2023), Thessaloniki, Greece, September 18th to 21st, 2023, CEUR Workshop Proceed… find
- Ramy Baly, Giovanni Da San Martino, James Glass, and Preslav Nakov. 2020a. We can detect your bias: Predicting the political ideology of news articles. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP’20, pages 4982–4991, Online. Association for Comput…
- Ramy Baly, Georgi Karadzhov, Dimitar Alexandrov, James Glass, and Preslav Nakov. 2018. Predicting factuality of reporting and bias of news media sources. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, EMNLP’18, pages 3528–3539, Brussels, Belgium. Associati…
- Ramy Baly, Georgi Karadzhov, Jisun An, Haewoon Kwak, Yoan Dinkov, Ahmed Ali, James Glass, and Preslav Nakov. 2020b. What was written vs. who read it: News media profiling using text analysis and social media context. In Proceedings of the 58th Annual Meeting of Assoc. for Comp. Linguistics, ACL’20,… find
- Ramy Baly, Georgi Karadzhov, Abdelrhman Saleh, James Glass, and Preslav Nakov. 2019. Multi-task ordinal regression for jointly predicting the trustworthiness and the leading political ideology of news media. In Proceedings of the 2019 Conference of the North American Chapter of the Assoc. for Comp.… find
- Alberto Barrón-Cedeño, Firoj Alam, Tommaso Caselli, Giovanni Da San Martino, Tamer Elsayed, Andrea Galassi, Fatima Haouari, Federico Ruggeri, Julia Maria Struß, Rabindra Nath Nandi, Gullal S. Cheema, Dilshod Azizov, and Preslav Nakov. 2023a. The CLEF-2023 checkthat! lab: Checkworthiness, subjectivit… find
- Alberto Barrón-Cedeño, Firoj Alam, Andrea Galassi, Giovanni Da San Martino, Preslav Nakov, Tamer Elsayed, Dilshod Azizov, Tommaso Caselli, Gullal S. Cheema, Fatima Haouari, Maram Hasanain, Mücahid Kutlu, Chengkai Li, Federico Ruggeri, Julia Maria Struß, and Wajdi Zaghouani. 2023b. Overview of the CL… find
- Cristian Bodnar, Fabrizio Frasca, Yuguang Wang, Nina Otter, Guido F. Montúfar, Pietro Lió, and Michael M. Bronstein. 2021. Weisfeiler and lehman go topological: Message passing simplicial networks. In Proceedings of the 38th International Conference on Machine Learning, ICML 2021, 18-24 July 2021, V… find
- Michael P Boyle, Mike Schmierbach, and Douglas M McLeod. 2007. Ideology, issues, and limited information: Implications for voting behavior. Atlantic Journal of Communication, 15(4):284–302. find
- Xavier Bresson and Thomas Laurent. 2017. Residual gated graph ConvNets. arXiv preprint arXiv:1711.07553. — [arXiv:1711.07553](https://arxiv.org/abs/1711.07553) find
- Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chri… — Language Models are Few-Shot Learners
- James Fairbanks, Natalie Fitch, Nathan Knauf, and Erica Briscoe. 2018. Credibility assessment in the news: do we need to read? In Proceedings of the MIS2 Workshop on 11th Int. Conf. on Web Search and Data Mining, WSDM’18, pages 799–800. ACM. find
- Sergio Burdisso, Dairazalia Sanchez-cortes, Esaú Villatoro-tello, and Petr Motlicek. 2024. Reliability estimation of news media sources: Birds of a feather flock together. In Proceedings of the 2024 Conference of the North American Chapter of the Assoc. for Comp. Linguistics: Human Language Technolo… find
- Xiaobo Guo, Weicheng Ma, and Soroush Vosoughi. 2022. Measuring media bias via masked language modeling. In Proceedings of the International AAAI Conference on Web and Social Media, ICWSM’22, pages 1404–1408. find
- Sonia Castelo, Thais G. Almeida, Anas Elghafari, Aécio S. R. Santos, Kien Pham, Eduardo Freire Nakamura, and Juliana Freire. 2019. A topic-agnostic approach for identifying fake news pages. In Companion of The 2019 World Wide Web Conference, WWW’19, pages 975–980, San Francisco, CA, USA. ACM. find
- Nadia K Conroy, Victoria L Rubin, and Yimin Chen. 2015. Automatic deception detection: Methods for finding fake news. Proceedings of the Association for Information Science and Technology, 52(1):1–4. find
- Ganqu Cui, Jie Zhou, Cheng Yang, and Zhiyuan Liu. 2020. Adaptive graph encoder for attributed graph embedding. In Proceedings of the 26th International Conference on Knowledge Discovery & Data Mining, KDD ’20, page 976–985, Virtual Event, CA, USA. Association for Computing Machinery. find
- Giovanni Da San Martino, Firoj Alam, Maram Hasanain, Rabindra Nath Nandi, Dilshod Azizov, and Preslav Nakov. 2023. Overview of the CLEF-2023 Check-That! lab task 3 on political bias of news articles and news media. In Working Notes of CLEF 2023–Conference and Labs of the Evaluation Forum, CLEF ’2023… find
- Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Assoc. for Comp. Linguistics: Human Language Technologies, Volume 1 (Lo…
- Kaize Ding, Zhe Xu, Hanghang Tong, and Huan Liu. 2022. Data augmentation for deep graph learning: A survey. SIGKDD Explorations Newsletter, 24(2):61– 77. find
- James N Druckman and Michael Parkin. 2005. The impact of media bias: How editorial slant affects voters. The Journal of Politics, 67(4):1030–1049. find
- Yoav Freund and Robert E Schapire. 1997. A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences, 55(1):119–139. find
- Will Hamilton, Zhitao Ying, and Jure Leskovec. 2017. Inductive representation learning on large graphs. In Proceedings of the 31st International Conference on Neural Information Processing Systems, volume 30 of NeurIPS’17, page 1025–1035, Long Beach, CA, USA. Curran Associates Inc.
- Marti A. Hearst, Susan T Dumais, Edgar Osuna, John Platt, and Bernhard Scholkopf. 1998. Support vector machines. IEEE Intelligent Systems and their applications, 13(4):18–28. find
- Austin Hounsel, Jordan Holland, Ben Kaiser, Kevin Borgolte, Nick Feamster, and Jonathan Mayer. 2020. Identifying disinformation websites using infrastructure features. In Proceedings of the 10th USENIX Workshop on Free and Open Communications on the Internet, FOCI’20, Virtual. find
- Edward J Hu, yelong shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. LoRA: Low-rank adaptation of large language models. In Proceedings of the International Conference on Learning Representations, ICLR ’22, virtual.
- Noussaiba Jaafar and Zied Lachiri. 2023. Multimodal fusion methods with deep neural networks and meta-information for aggression detection in surveillance. Expert Systems with Applications, 211:118523. find
- Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. 2023. Mistral 7B. arXiv preprint arXiv:2310.06825. find
- Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Soricut. 2020. ALBERT: A lite BERT for self-supervised learning of language representations. In Proceedings of the International Conference on Learning Representations, ICLR’20.
- Yuanyuan Lei and Ruihong Huang. 2024. Sentence-level media bias analysis with event relation graph. In Proceedings of the 2024 Conference of the North American Chapter of the Assoc. for Comp. Linguistics: Human Language Technologies (Volume 1: Long Papers), NAACL’24, pages 5225–5238, Mexico City, Me… find
- Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020. BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. In Proceedings of the 58th Annual Meeting of the…
- Nikhil Mehta and Dan Goldwasser. 2023. An interactive framework for profiling news media sources. arXiv preprint arXiv:2309.07384. find
- Luyang Lin, Lingzhi Wang, Xiaoyan Zhao, Jing Li, and Kam-Fai Wong. 2024. IndiVec: An exploration of leveraging large language models for media bias detection with fine-grained bias indicators. In Findings of the Assoc. for Comp. find
- Linguistics: EACL 2024, pages 1038–1050, St. Julian’s, Malta. Association for Computational Linguistics. find
- Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. RoBERTa: A robustly optimized BERT pretraining approach. arXiv preprint arXiv:1907.11692, abs/1907.11692.
- Yujian Liu, Xinliang Frederick Zhang, David Wegsman, Nicholas Beauchamp, and Lu Wang. 2022. POLI-TICS: Pretraining with same-story article comparison for ideology prediction and stance detection. In Findings of the Assoc. for Comp. find
- Linguistics: NAACL 2022, Seattle, United States. Association for Computational Linguistics. find
- Antonio Longa, Steve Azzolin, Gabriele Santin, Giulia Cencetti, Pietro Liò, Bruno Lepri, and Andrea Passerini. 2024. Explaining the explainers in graph neural networks: a comparative study. ACM Computing Surveys. find
- Hehuan Ma, Yatao Bian, Yu Rong, Wenbing Huang, Tingyang Xu, Weiyang Xie, Geyan Ye, and Junzhou Huang. 2020. Multi-view graph neural networks for molecular property prediction. arXiv preprint arXiv:2005.13607. — [arXiv:2005.13607](https://arxiv.org/abs/2005.13607) find
- Iffat Maab, Edison Marrese-Taylor, Sebastian Padó, and Yutaka Matsuo. 2024. Media bias detection across families of language models. In Proceedings of the 2024 Conference of the North American Chapter of the Assoc. for Comp. Linguistics: Human Language Technologies (Volume 1: Long Papers), NAACL’24,… find
- Muhammad Arslan Manzoor, Ruihong Zeng, Dilshod Azizov, Preslav Nakov, and Shangsong Liang. 2025. MGM: Global understanding of audience overlap graphs for predicting the factuality and the bias of news media. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Assoc. find
- Nikhil Mehta, Maria Leonor Pacheco, and Dan Goldwasser. 2022. Tackling fake news detection by continually improving social context representations using graph neural networks. In Proceedings of the 60th Annual Meeting of the Assoc. for Comp. Linguistics (Volume 1: Long Papers), ACL’22, pages 1363–13… find
- Christopher Morris, Martin Ritzert, Matthias Fey, William L. Hamilton, Jan Eric Lenssen, Gaurav Rattan, and Martin Grohe. 2019. Weisfeiler and Leman go neural: higher-order graph neural networks. In Proceedings of the Thirty-Third Conference on Artificial Intelligence, AAAI’19, Honolulu, Hawaii, USA… find
- Zain Muhammad Mujahid, Dilshod Azizov, Maha Tufail Agro, and Preslav Nakov. 2025. Profiling news media for factuality and bias using LLMs and the fact-checking methodology of human experts. In Findings of the Assoc. for Comp. find
- Linguistics: ACL 2025, pages 798–819, Vienna, Austria. ACL. find
- Subhabrata Mukherjee and Gerhard Weikum. 2015. Leveraging joint interactions for credibility analysis in news communities. In Proceedings of the 24th International Conference on Information and Knowledge Management, CIKM’15, pages 353–362, Melbourne, VIC, Australia. ACM. find
- Preslav Nakov, Jisun An, Haewoon Kwak, Muhammad Arslan Manzoor, Zain Muhammad Mujahid, and Husrev Taha Sencar. 2024. A survey on predicting the factuality and the bias of news media. In Findings of the Assoc. for Comp. find
- Linguistics: ACL 2024, pages 15947–15962, Bangkok, Thailand. Association for Computational Linguistics. find
- Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul F Christiano, Jan Leike, and Ryan Lowe. 2022. Tra… — Training language models to follow instructions with human feedback
- Panayot Panayotov, Utsav Shukla, Husrev Taha Sencar, Mohamed Nabeel, and Preslav Nakov. 2022. GREENER: Graph neural networks for news media profiling. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, EMNLP’22, pages 7470–7480, Abu Dhabi, United Arab Emirates… find
- Verónica Pérez-Rosas, Bennett Kleinberg, Alexandra Lefevre, and Rada Mihalcea. 2018. Automatic detection of fake news. In Proceedings of the 27th International Conference on Computational Linguistics, CCL’18, pages 3391–3401, Santa Fe, New Mexico, USA. ACL.
- Kashyap Popat, Subhabrata Mukherjee, Jannik Strötgen, and Gerhard Weikum. 2017. Where the truth lies: Explaining the credibility of emerging claims on the Web and social media. In WWW Companion, pages 1003–1012, Perth, Australia. find
- Markus Prior. 2013. Media and political polarization. Annual review of political science, 16(1):101–127. find
- Changyuan Qiu, Winston Wu, Xinliang Frederick Zhang, and Lu Wang. 2022. Late fusion with triplet margin objective for multimodal ideology prediction and analysis. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, EMNLP’22, pages 9720–9736, Abu Dhabi, United A… find
- Antonin Raffin, Ashley Hill, Adam Gleave, Anssi Kanervisto, Maximilian Ernestus, and Noah Dormann. 2021. Stable-Baselines3: Reliable reinforcement learning implementations. Journal of Machine Learning Research, 22(268):1–8. find
- Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. 2019. DistilBERT, a distilled version of bert: smaller, faster, cheaper and lighter. arXiv preprint arXiv:1910.01108. find
- John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347. find
- Karen Sparck Jones. 1972. A statistical interpretation of term specificity and its application in retrieval. Journal of documentation, 28(1):11–21. find
- Ke Sun, Zhouchen Lin, and Zhanxing Zhu. 2020. Multi-stage self-supervised learning for graph convolutional networks on graphs with few labeled nodes. In Proceedings of the Conference on Artificial Intelligence, volume 34 of AAAI’20, pages 5892–5899. find
- Nurbek Tastan, Stefanos Laskaridis, Karthik Nandakumar, and Samuel Horvath. 2026a. MoSE: Mixture of slimmable experts for efficient and adaptive language models. arXiv preprint arXiv:2602.06154. — [arXiv:2602.06154](https://arxiv.org/abs/2602.06154) find
- Nurbek Tastan, Stefanos Laskaridis, Martin Takáč, Karthik Nandakumar, and Samuel Horváth. 2026b. LoFT: Low-rank adaptation that behaves like full fine-tuning. In The Fourteenth Int. Conf. on Learning Representations, ICLR’26, Rio de Janeiro, Brazil. find
- Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023. LLaMA: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971.
- Mark Towers, Ariel Kwiatkowski, Jordan K. Terry, John U. Balis, Gianluca De Cola, Tristan Deleu, Manuel Goulão, Andreas Kallinteris, Markus Krimmel, Arjun KG, Rodrigo Perez-Vicente, Andrea Pierré, Sander Schulhoff, Jun Jet Tai, Hannah Tan, and Omar G. Younis. 2024. Gymnasium: A standard interface fo… find
- Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V. Le, and Denny Zhou. 2022. Chain-of-thought prompting elicits reasoning in large language models. In Proceedings of the 36th International Conference on Neural Information Processing Systems, NeurIPS ’22…
- Kai-Cheng Yang and Filippo Menczer. 2025. Accuracy and political bias of news source credibility ratings by large language models. In Proceedings of the 17th ACM Web Science Conference, Websci ’25, page 127–137, New Brunswick, New Jersey, USA. ACM. find
- Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Thomas L. Griffiths, Yuan Cao, and Karthik Narasimhan. 2023. Tree of thoughts: deliberate problem solving with large language models. In Proceedings of the 37th International Conference on Neural Information Processing Systems, NeurIPS ’23, New Orlea… find
- Jinliang Yuan, Hualei Yu, Meng Cao, Ming Xu, Junyuan Xie, and Chongjun Wang. 2021. Semi-supervised and self-supervised classification with multi-view graph neural networks. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management, CIKM ’21, page 2466–2476, Virtua… find
- Jiahao Zhang, Rui Xue, Wenqi Fan, Xin Xu, Qing Li, Jian Pei, and Xiaorui Liu. 2024. Linear-time graph neural networks for scalable recommendations. In Proceedings of the ACM Web Conference 2024, WWW ’24, page 3533–3544, Singapore, Singapore. Association for Computing Machinery. find
- Wenqian Zhang, Shangbin Feng, Zilong Chen, Zhenyu Lei, Jundong Li, and Minnan Luo. 2022. KCD: Knowledge walks and textual cues enhanced political perspective detection in news media. In Proceedings of the 2022 Conference of the North American Chapter of the Assoc. for Comp. Linguistics: Human Langua… find
- Denny Zhou, Nathanael Schärli, Le Hou, Jason Wei, Nathan Scales, Xuezhi Wang, Dale Schuurmans, Claire Cui, Olivier Bousquet, and Quoc V. Le. 2023. Least-to-most prompting enables complex reasoning in large language models. In Proceedings of the International Conference on Learning Representations, I…
Updated
This page in Markdown → https://md.research.elamur.ai/free/en/papers/manzoor-2026-multi-view-media-profiling-suite/