Elamur Research⌕
☰
home / papers / 2026 / AFMRL: Attribute-Enhanced Fine-Grained Multi-Modal Representation Learning in E-commerce

AFMRL: Attribute-Enhanced Fine-Grained Multi-Modal Representation Learning in E-commerce

Paper · 2026 · Biao Zhang, Lixin Chen, Bin Zhang et al.

Summary

Multimodal embedding models built on large vision-language backbones, such as VLM2Vec, understand products well in general. They are weaker at the fine-grained details that separate near-identical e-commerce items, and their pooled or last-token embeddings do not fit region-level alignment methods. The authors, from Alibaba's Taobao & Tmall Group, treat fine-grained understanding as a task of generating key product attributes with an MLLM.

AFMRL trains in two stages:

  • AGCL: attributes from a generator distilled from Qwen2.5-VL-72B-Instruct guide contrastive training of a VLM2Vec-based encoder. BM25 overlap between attribute sets up-weights hard negatives, and overly similar in-batch candidates are masked as probable false negatives.
  • RAR: the frozen encoder supplies a Recall@k reward, and GRPO uses it to fine-tune the attribute generator.

At inference, the generated attributes are added to the query before it is encoded. The authors report state-of-the-art results on several large-scale e-commerce retrieval tasks. This suggests that generative reasoning can sharpen discriminative retrieval embeddings.

Key points

  • Loss design: BM25 scores are turned into bounded weights through a tanh-based exponential. Negatives whose cosine similarity exceeds the positive's by a margin δ are excluded from the InfoNCE denominator.
  • Reward choice: k = 50 worked best for the Recall@k reward. Smaller k (10) gave rewards that were too sparse, and larger k (100) gave rewards that saturated.
  • Emergent conciseness: during RL, the generated attribute lists became steadily shorter, because redundant attributes acted as noise that lowered recall.
  • Circular Iterative Training (CIT): the RL-tuned generator was fed back into AGCL using only 30% of the training data. On downstream tasks, this raised Accuracy from 87.80 (SFT) to 89.13 and ARI from 44.44 to 47.40.
  • Stated limitation: the authors note an "alignment tax." Optimizing for Recall@k specializes the representations, so the RL policy's gains on classification and clustering are only marginal over SFT.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🏢Alibaba GroupHangzhou · China201 works · 17,173 citations

 Fields, topics and concepts

📚 Shelves 3The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksComputer Vision and Pattern Recognition · 3,486 worksInformation Systems · 1,730 works

🧭 Topics 1Research subjects and languages the work is about.Vision & language · 811 works

Cite

Cite as Biao Zhang, Lixin Chen, Bin Zhang, Wang Zongwei, Tong Liu and Bo Zheng. 2026. AFMRL: Attribute-Enhanced Fine-Grained Multi-Modal Representation Learning in E-commerce. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.704

@inproceedings{zhang2026afmrl,
  title = {{AFMRL: Attribute-Enhanced Fine-Grained Multi-Modal Representation Learning in E-commerce}},
  author = {Biao Zhang, Lixin Chen, Bin Zhang, Wang Zongwei, Tong Liu and Bo Zheng},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.704},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.704}
}

References

30 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, and 8 others. 2025. Qwen2.5-vl technical report. arXiv preprint arXiv:25…
  2. Parishad BehnamGhader, Vaibhav Adlakha, Marius Mosbach, Dzmitry Bahdanau, Nicolas Chapados, and Siva Reddy. 2024. Llm2vec: Large language models are secretly powerful text encoders. arXiv preprint arXiv:2404.05961.
  3. Xiao Dong, Xunlin Zhan, Yangxin Wu, Yunchao Wei, Michael C Kampffmeyer, Xiaoyong Wei, Minlong Lu, Yaowei Wang, and Xiaodan Liang. 2022. M5product: Self-harmonized contrastive learning for e-commercial multi-modal pretraining. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern R… find
  4. Luyu Gao, Yunyi Zhang, Jiawei Han, and Jamie Callan. 2021. Scaling deep contrastive learning batch size under memory limited setup. In Proceedings of the 6th Workshop on Representation Learning for NLP (RepL4NLP-2021), pages 316–321.
  5. Raia Hadsell, Sumit Chopra, and Yann LeCun. 2006. Dimensionality reduction by learning an invariant mapping. In 2006 IEEE computer society conference on computer vision and pattern recognition (CVPR’06), volume 2, pages 1735–1742. IEEE. find
  6. Weiquan Huang, Aoqi Wu, Yifan Yang, Xufang Luo, Yuqing Yang, Liang Hu, Qi Dai, Chunyu Wang, Xiyang Dai, Dongdong Chen, and 1 others. 2024. Llm2clip: Powerful language model unlocks richer visual representation. arXiv preprint arXiv:2411.04997. — [arXiv:2411.04997](https://arxiv.org/abs/2411.04997) find
  7. Ting Jiang, Minghui Song, Zihan Zhang, Haizhen Huang, Weiwei Deng, Feng Sun, Qi Zhang, Deqing Wang, and Fuzhen Zhuang. 2024. E5-v: Universal embeddings with multimodal large language models. arXiv preprint arXiv:2407.12580. — [arXiv:2407.12580](https://arxiv.org/abs/2407.12580) find
  8. Ziyan Jiang, Rui Meng, Xinyi Yang, Semih Yavuz, Yingbo Zhou, and Wenhu Chen. 2025. Vlm2vec: Training vision-language models for massive multimodal embedding tasks. In ICLR. find
  9. Yang Jin, Yongzhi Li, Zehuan Yuan, and Yadong Mu. 2023. Learning instance-level representation for large-scale multi-modal pretraining in e-commerce. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11060– 11069. find
  10. Chankyu Lee, Rajarshi Roy, Mengyao Xu, Jonathan Raiman, Mohammad Shoeybi, Bryan Catanzaro, and Wei Ping. 2024. Nv-embed: Improved techniques for training llms as generalist embedding models. arXiv preprint arXiv:2405.17428.
  11. Junjie Li, Jianghong Ma, Xiaofeng Zhang, Yuhang Li, and Jianyang Shi. 2024. Give: Guiding visual encoder to perceive overlooked information. arXiv preprint arXiv:2410.20109. — [arXiv:2410.20109](https://arxiv.org/abs/2410.20109) find
  12. Junnan Li, Ramprasaath Selvaraju, Akhilesh Gotmare, Shafiq Joty, Caiming Xiong, and Steven Chu Hong Hoi. 2021. Align before fuse: Vision and language representation learning with momentum distillation. Advances in neural information processing systems, 34:9694–9705. find
  13. Siting Li, Xiang Gao, and Simon Shaolei Du. 2025. Highlighting what matters: Promptable embeddings for attribute-focused image retrieval. arXiv preprint arXiv:2505.15877. — [arXiv:2505.15877](https://arxiv.org/abs/2505.15877) find
  14. Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2023. Visual instruction tuning. Advances in neural information processing systems, 36:34892– 34916.
  15. Haoyu Lu, Wen Liu, Bo Zhang, Bingxuan Wang, Kai Dong, Bo Liu, Jingxiang Sun, Tongzheng Ren, Zhuoshu Li, Hao Yang, and 1 others. 2024. Deepseek-vl: towards real-world vision-language understanding. arXiv preprint arXiv:2403.05525.
  16. Mert Yuksekgonul, Federico Bianchi, Pratyusha Kalluri, Dan Jurafsky, and James Zou. 2022. When and why vision-language models behave like bags-of-words, and what to do about it? arXiv preprint arXiv:2210.01936. find
  17. Xueguang Ma, Liang Wang, Nan Yang, Furu Wei, and Jimmy Lin. 2024. Fine-tuning llama for multi-stage text retrieval. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 2421– 2425.
  18. Chao Zhang, Shiwei Wu, Haoxin Zhang, Tong Xu, Yan Gao, Yao Hu, and Enhong Chen. 2024a. Notellm: A retrievable large language model for note recommendation. In Companion Proceedings of the ACM Web Conference 2024, pages 170–179. find
  19. Daiqing Qi, Handong Zhao, and Sheng Li. 2024. Easy regional contrastive learning of expressive fashion representations. Advances in Neural Information Processing Systems, 37:20480–20509. find
  20. Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, and 1 others. 2021. Learning transferable visual models from natural language supervision. In International conference on machine learning, pages 8748–8…
  21. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, and 1 others. 2024. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300.
  22. JungKyoo Shin, Bumsoo Kim, and Eunwoo Kim. 2025. Generative modeling of class probability for multimodal representation learning. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 20737–20746. find
  23. Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, and 1 others. 2024a. Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution. arXiv preprint arXiv:2409.12191.
  24. Kai Zhang, Yi Luan, Hexiang Hu, Kenton Lee, Siyuan Qiao, Wenhu Chen, Yu Su, and Ming-Wei Chang. 2024b. Magiclens: self-supervised image retrieval with open-ended instructions. In Proceedings of the 41st International Conference on Machine Learning, pages 59403–59420. find
  25. Peitian Zhang, Shitao Xiao, Zheng Liu, Zhicheng Dou, and Jian-Yun Nie. 2023. Retrieve anything to augment large language models. arXiv preprint arXiv:2310.07554. — [arXiv:2310.07554](https://arxiv.org/abs/2310.07554) find
  26. Xin Zhang, Yanzhao Zhang, Wen Xie, Mingxin Li, Ziqi Dai, Dingkun Long, Pengjun Xie, Meishan Zhang, Wenjie Li, and Min Zhang. 2025. Bridging modalities: Improving universal multimodal retrieval by multimodal large language models. In Proceedings of the Computer Vision and Pattern Recognition Conferen… find
  27. Junjie Zhou, Yongping Xiong, Zheng Liu, Ze Liu, Shitao Xiao, Yueze Wang, Bo Zhao, Chen Jason Zhang, and Defu Lian. 2025. MegaPairs: Massive data synthesis for universal multimodal retrieval. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Pa…
  28. Weihan Wang, Qingsong Lv, Wenmeng Yu, Wenyi Hong, Ji Qi, Yan Wang, Junhui Ji, Zhuoyi Yang, Lei Zhao, Song XiXuan, and 1 others. 2024b. Cogvlm: Visual expert for pretrained language models. Advances in Neural Information Processing Systems, 37:121475–121499.
  29. Rui Xiao, Sanghwan Kim, Mariana-Iuliana Georgescu, Zeynep Akata, and Stephan Alaniz. 2025. Flair: Vlm with fine-grained language-informed image representations. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 24884– 24894. find
  30. Chunyu Xie, Bin Wang, Fanjing Kong, Jincheng Li, Dawei Liang, Gengshen Zhang, Dawei Leng, and Yuhui Yin. 2025. Fg-clip: Fine-grained visual and textual alignment. arXiv preprint arXiv:2505.05071. — [arXiv:2505.05071](https://arxiv.org/abs/2505.05071) find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zhang-2026-afmrl-attribute-enhanced-fine-grained/