Elamur Research⌕
☰
home / papers / 2025 / IMOL: Incomplete-Modality-Tolerant Learning for Multi-Domain Fake News Video Detection

IMOL: Incomplete-Modality-Tolerant Learning for Multi-Domain Fake News Video Detection

Paper · 2025 · Zhi Zeng, Jiaying Wu, Minnan Luo et al.

Summary

Detectors for fake news videos usually assume that every post has video, audio, text and an image, and they are usually trained on a single topic area. Real platforms break both assumptions: posts cover many domains, and modalities are often lost through expired resources, corruption or empty fields. The authors build two benchmarks, FakeSV IM and FakeTT IM, from the existing FakeSV and FakeTT datasets. They randomly remove combinations of modalities and add labels for nine news domains.

They also propose IMOL, a framework inspired by how humans fill in missing senses and recall similar memories. IMOL works in two stages:

  • It uses cross-modal consistency learning to reconstruct the absent modalities.
  • It uses retrieval-augmented contrastive learning, which draws on related samples to keep knowledge that transfers across domains.

IMOL beats multimodal baselines such as SV-FEND, MMVD and MoMKE at every tested missing rate. It also generalizes better to unseen domains and later time periods, and it outperforms zero-shot GPT models. This matters for building misinformation filters that hold up when real-world data is messy.

Key points

  • Benchmark design: Each sample has four modalities, so there are 15 possible missing-modality patterns. At least one modality is always kept. The nine domain labels span finance, education and the military, along with science, politics and entertainment, plus health, society and disasters.
  • Ablation results (missing rate 0.7): Full IMOL reaches 77.19 ACC / 77.12 F1 on FakeSV IM and 79.23 ACC / 78.36 F1 on FakeTT IM. Removing domain information hurts most, cutting FakeTT IM accuracy by 2.68 points.
  • Efficiency: IMOL uses 718.06M trainable parameters and 11.87G FLOPs, fewer parameters than MoMKE.
  • Comparison with LLMs on unseen domains: Models were trained on Ch-6/En-6 and tested on Ch-3/En-3. On En-3, IMOL reaches 70.20 ACC, compared with 57.48 for GPT-4V and 50.89 for GPT-3.5-turbo. On Ch-3, IMOL reaches 71.58 ACC versus 64.63 for GPT-4V.
  • Temporal robustness: The data was split 8:2 by publication date. IMOL again outperformed the baselines on the later videos.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓National University of SingaporeSingapore · Singapore180 works · 34,304 citations🎓Xi'an Jiaotong UniversityXi'an · China22 works · 36,046 citations🏢State Grid Corporation of ChinaBeijing · China2 works · 23 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksComputer Vision and Pattern Recognition · 2,869 worksSocial Sciences · 1,772 worksInformation Systems · 1,437 worksSociology and Political Science · 577 works

Cite

Cite as Zhi Zeng, Jiaying Wu, Minnan Luo, Herun Wan, Xiangzheng Kong, Zihan Ma, Guang Dai and Qinghua Zheng. 2025. IMOL: Incomplete-Modality-Tolerant Learning for Multi-Domain Fake News Video Detection. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2025.acl-long.1494

@inproceedings{zeng2025imol,
  title = {{IMOL: Incomplete-Modality-Tolerant Learning for Multi-Domain Fake News Video Detection}},
  author = {Zhi Zeng, Jiaying Wu, Minnan Luo, Herun Wan, Xiangzheng Kong, Zihan Ma, Guang Dai and Qinghua Zheng},
  booktitle = {Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2025},
  doi = {10.18653/v1/2025.acl-long.1494},
  url = {https://doi.org/10.18653/v1/2025.acl-long.1494}
}

References

54 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774.
  2. Jinhe Bi, Yifan Wang, Danqi Yan, Xun Xiao, Artur Hecker, Volker Tresp, and Yunpu Ma. 2025a. Prism: Self-pruning intrinsic selection method for training-free multimodal data selection. arXiv preprint arXiv:2502.12119. — [arXiv:2502.12119](https://arxiv.org/abs/2502.12119) find
  3. Jinhe Bi, Yujun Wang, Haokun Chen, Xun Xiao, Artur Hecker, Volker Tresp, and Yunpu Ma. 2024. Visual instruction tuning with 500x fewer parameters through modality linear representation-steering. arXiv preprint arXiv:2412.12359. — [arXiv:2412.12359](https://arxiv.org/abs/2412.12359) find
  4. Jinhe Bi, Danqi Yan, Yifan Wang, Wenke Huang, Haokun Chen, Guancheng Wan, Mang Ye, Xun Xiao, Hinrich Schuetze, Volker Tresp, et al. 2025b. Cot-kinetics: A theoretical modeling assessing lrm reasoning process. arXiv preprint arXiv:2505.13408. — [arXiv:2505.13408](https://arxiv.org/abs/2505.13408) find
  5. Yuyan Bu, Qiang Sheng, Juan Cao, Peng Qi, Danding Wang, and Jintao Li. 2024. Fakingrecipe: Detecting fake news on short video platforms from the perspective of creative process. arXiv preprint arXiv:2407.16670. — [arXiv:2407.16670](https://arxiv.org/abs/2407.16670) find
  6. Lei Cai, Zhengyang Wang, Hongyang Gao, Dinggang Shen, and Shuiwang Ji. 2018. Deep adversarial learning for multi-modality missing data completion. In Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining, pages 1158–1166. find
  7. Hyewon Choi and Youngjoong Ko. 2021. Using topic modeling and adversarial neural networks for fake news video detection. In Proceedings of the 30th ACM international conference on information & knowledge management, pages 2950–2954. find
  8. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805.
  9. Guodong Du, Zitao Fang, Jing Li, Junlin Li, Runhua Jiang, Shuyang Yu, Yifei Guo, Yangneng Chen, Sim Kuan Goh, Ho-Kin Tang, Daojing He, Honghai Liu, and Min Zhang. 2025a. Neural parameter search for slimmer fine-tuned models and better transfer. arXiv preprint arXiv:2505.18713.
  10. Guodong Du, Xuanning Zhou, Junlin Li, Zhuo Li, Zesheng Shi, Wanyu Lin, Ho-Kin Tang, Xiucheng Li, Fangming Liu, Wenya Wang, Min Zhang, and Jing Li. 2025b. Knowledge grafting of large language models. arXiv preprint arXiv:2505.18502. — [arXiv:2505.18502](https://arxiv.org/abs/2505.18502) find
  11. Lifang Fu, Huanxin Peng, and Shuai Liu. 2023. Kg-mfend: an efficient knowledge graph-based model for multi-domain fake news detection. The Journal of Supercomputing, 79(16):18417–18444. find
  12. Tianyu Gao, Xingcheng Yao, and Danqi Chen. 2021. Simcse: Simple contrastive learning of sentence embeddings. arXiv preprint arXiv:2104.08821.
  13. Zirun Guo, Tao Jin, and Zhou Zhao. 2024. Multimodal prompt learning with missing modalities for sentiment analysis and emotion recognition. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1726–1736. find
  14. Shawn Hershey, Sourish Chaudhuri, Daniel PW Ellis, Jort F Gemmeke, Aren Jansen, R Channing Moore, Manoj Plakal, Devin Platt, Rif A Saurous, Bryan Seybold, et al. 2017. Cnn architectures for large-scale audio classification. In 2017 ieee international conference on acoustics, speech and signal proces… find
  15. Rui Hou, Verónica Pérez-Rosas, Stacy Loeb, and Rada Mihalcea. 2019. Towards automatic detection of misinformation in online medical videos. In 2019 International conference on multimodal interaction, pages 235–243. find
  16. Jiayang Li, Xuan Feng, Tianlong Gu, and Liang Chang. 2024. Dual-teacher de-biasing distillation framework for multi-domain fake news detection. In 2024 IEEE 40th International Conference on Data Engineering (ICDE), pages 3627–3639. IEEE. find
  17. Xiang Li, Chaofan Fu, Zhongying Zhao, Guanjie Zheng, Chao Huang, Yanwei Yu, and Junyu Dong. 2025. Dual-channel multiplex graph neural networks for recommendation. IEEE Transactions on Knowledge and Data Engineering. find
  18. Xiaojun Li, Xvhao Xiao, Jia Li, Changhua Hu, Jun-ping Yao, and Shaochen Li. 2022. A cnn-based misleading video detection model. Scientific Reports, 12(1):6092. find
  19. Zheng Lian, Lan Chen, Licai Sun, Bin Liu, and Jianhua Tao. 2023. Gcnet: Graph completion network for incomplete multimodal learning in conversation. IEEE Transactions on pattern analysis and machine intelligence, 45(7):8419–8432. find
  20. Fuxiao Liu, Yaser Yacoob, and Abhinav Shrivastava. 2023. Covid-vts: Fact extraction and verification on short video platforms. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 178–188. find
  21. Rui Liu, Haolin Zuo, Zheng Lian, Bjorn W Schuller, and Haizhou Li. 2024. Contrastive learning based modality-invariant feature acquisition for robust multimodal emotion recognition with missing modalities. IEEE Transactions on Affective Computing. find
  22. Zihan Ma, Minnan Luo, Hao Guo, Zhi Zeng, Yiran Hao, and Xiang Zhao. 2024. Event-radar: Event-driven multi-view learning for multimodal fake news detection. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 5809–5821. find
  23. Jingbiao Mei, Jinghong Chen, Weizhe Lin, Bill Byrne, and Marcus Tomalin. 2024. Improving hateful meme detection through retrieval-guided contrastive learning. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 5333–5347. find
  24. Qiong Nan, Juan Cao, Yongchun Zhu, Yanyan Wang, and Jintao Li. 2021. Mdfend: Multi-domain fake news detection. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management, pages 3343–3347. find
  25. Qiong Nan, Qiang Sheng, Juan Cao, Yongchun Zhu, Danding Wang, Guang Yang, and Jintao Li. 2023. Exploiting user comments for early detection of fake news prior to users’ commenting. arXiv preprint arXiv:2310.10429. — [arXiv:2310.10429](https://arxiv.org/abs/2310.10429) find
  26. A Nielsen. 2015. Neural networks and deep learning. find
  27. Priyank Palod, Ayush Patwari, Sudhanshu Bahety, Saurabh Bagchi, and Pawan Goyal. 2019. Misleading metadata detection on youtube. In Advances in Information Retrieval: 41st European Conference on IR Research, ECIR 2019, Cologne, Germany, April 14–18, 2019, Proceedings, Part II 41, pages 140–147. Spri… find
  28. Peng Qi, Yuyan Bu, Juan Cao, Wei Ji, Ruihao Shui, Junbin Xiao, Danding Wang, and Tat-Seng Chua. 2023a. Fakesv: A multimodal benchmark with rich social context for fake news detection on short video platforms. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 14444– 1… find
  29. Peng Qi, Yuyang Zhao, Yufeng Shen, Wei Ji, Juan Cao, and Tat-Seng Chua. 2023b. Two heads are better than one: Improving fake news video detection by correlating with neighbors. In Findings of the Association for Computational Linguistics: ACL 2023, pages 11947–11959. find
  30. Juan Carlos Medina Serrano, Orestis Papakyriakopoulos, and Simon Hegelich. 2020. Nlp-based feature extraction for the detection of covid-19 misinformation videos on youtube. In Proceedings of the 1st Workshop on NLP for COVID-19 at ACL 2020. find
  31. Lanyu Shang, Ziyi Kou, Yang Zhang, and Dong Wang. 2021. A multimodal misinformation detector for covid-19 short videos on tiktok. In 2021 IEEE international conference on big data (big data), pages 899–908. IEEE. find
  32. Guangyao Shen, Xin Wang, Xuguang Duan, Hongzhi Li, and Wenwu Zhu. 2020. Memor: A dataset for multimodal emotion reasoning in videos. In Proceedings of the 28th ACM international conference on multimedia, pages 493–502. find
  33. Karen Simonyan and Andrew Zisserman. 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556.
  34. Eleanor Spens and Neil Burgess. 2024. A generative model of memory construction and consolidation. Nature Human Behaviour, 8(3):526–543. find
  35. Yu Tong, Weihai Lu, Zhe Zhao, Song Lai, and Tong Shi. 2024. Mmdfnd: Multi-modal multi-domain fake news detection. In Proceedings of the 32nd ACM International Conference on Multimedia, pages 1178– 1186. find
  36. Du Tran, Lubomir Bourdev, Rob Fergus, Lorenzo Torresani, and Manohar Paluri. 2015. Learning spatiotemporal features with 3d convolutional networks. In Proceedings of the IEEE international conference on computer vision, pages 4489–4497. find
  37. Luan Tran, Xiaoming Liu, Jiayu Zhou, and Rong Jin. 2017. Missing modalities imputation via cascaded residual autoencoder. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1405–1414. find
  38. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS’17, page 6000–6010, Red Hook, NY, USA. Cur…
  39. Federated long-tailed learning with self-adjusting gradient balancer. In Advances in Neural Information Processing Systems, volume 36, pages 77745–77757. Curran Associates, Inc. find
  40. Zikai Xiao, Ziyang Wang, Wen Ma, Yan Zhang, Wei Shen, Yan Wang, Luqi Gong, and Zuozhu Liu. 2025. Mitigating posterior salience attenuation in long-context llms with positional contrastive decoding. arXiv preprint arXiv:2506.08371.
  41. Wenxin Xu, Hexin Jiang, and Xuefeng Liang. 2024. Leveraging knowledge of modality experts for incomplete multimodal learning. In Proceedings of the 32nd ACM International Conference on Multimedia, pages 438–446. find
  42. Ziqi Yuan, Wei Li, Hua Xu, and Wenmeng Yu. 2021. Transformer-based feature reconstruction network for robust multimodal sentiment analysis. In Proceedings of the 29th ACM International Conference on Multimedia, pages 4400–4407. find
  43. Zhi Zeng, Minnan Luo, Xiangzheng Kong, Huan Liu, Hao Guo, Hao Yang, Zihan Ma, and Xiang Zhao. 2024. Mitigating world biases: A multimodal multi-view debiasing framework for fake news video detection. In Proceedings of the 32nd ACM International Conference on Multimedia, pages 6492–6500. find
  44. Yake Wei, Di Hu, Yapeng Tian, and Xuelong Li. 2022. Learning in audio-visual context: A review, analysis, and new perspective. arXiv preprint arXiv:2208.09579. — [arXiv:2208.09579](https://arxiv.org/abs/2208.09579) find
  45. Jinming Zhao, Ruichen Li, and Qin Jin. 2021. Missing modality imagination network for emotion recognition with uncertain missing modalities. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Proc… find
  46. Jiaying Wu and Bryan Hooi. 2023. Decor: Degree-corrected social graph refinement for fake news detection. In Proceedings of the 29th ACM SIGKDD conference on knowledge discovery and data mining, pages 2582–2593. find
  47. Yongchun Zhu, Qiang Sheng, Juan Cao, Qiong Nan, Kai Shu, Minghui Wu, Jindong Wang, and Fuzhen Zhuang. 2022a. Memory-guided multi-view multi-domain fake news detection. IEEE Transactions on Knowledge and Data Engineering, 35(7):7178–7191. find
  48. Jiaying Wu, Fanxiao Li, Min-Yen Kan, and Bryan Hooi. 2025. Seeing through deception: Uncovering misleading creator intent in multimodal news with vision-language models. arXiv preprint arXiv:2505.15489. — [arXiv:2505.15489](https://arxiv.org/abs/2505.15489) find
  49. Yongchun Zhu, Qiang Sheng, Juan Cao, Qiong Nan, Kai Shu, Minghui Wu, Jindong Wang, and Fuzhen Zhuang. 2022b. Memory-guided multi-view multi-domain fake news detection. IEEE Transactions on Knowledge and Data Engineering, 35(7):7178–7191. find
  50. Mason Walker and Katerina Eva Matsa. 2021. News consumption across social media in 2021. find
  51. Jiaying Wu, Shen Li, Ailin Deng, Miao Xiong, and Bryan Hooi. 2023. Prompt-and-align: prompt-based social alignment for few-shot fake news detection. In Proceedings of the 32nd ACM International Conference on Information and Knowledge Management, pages 2726–2736. find
  52. Kaixuan Wu, Yanghao Lin, Donglin Cao, and Dazhen Lin. 2024. Interpretable short video rumor detection based on modality tampering. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 9180–9189. find
  53. Zikai Xiao, Zihan Chen, Songshang Liu, Hualiang Wang, YANG FENG, Jin Hao, Joey Tianyi Zhou, Jian Wu, Howard Yang, and Zuozhu Liu. 2023. find
  54. Fed-grab: Linlin Zong, Jiahui Zhou, Wenmin Lin, Xinyue Liu, Xianchao Zhang, and Bo Xu. 2024. Unveiling opinion evolution via prompting and diffusion for short video fake news detection. In Findings of the Association for Computational Linguistics ACL 2024, pages 10817–10826. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zeng-2025-imol-incomplete-modality-tolerant-learning/