Elamur Research⌕
☰
home / papers / 2026 / Tricking Open-World Object Recognition Models: Uncertainty in Out-of-Distribution Detection

Tricking Open-World Object Recognition Models: Uncertainty in Out-of-Distribution Detection

Paper · 2026 · Wout Teillers and Matias Valdenegro-Toro

Summary

Object detectors are usually benchmarked on whether they find objects that actually appear in an image. In deployment, however, a system rarely knows in advance whether a queried object is present. This paper probes that gap by giving three Open-World detectors (YOLO-World, Grounding DINO and GPT-4o) "impossible prompts", which ask for object classes absent from the image, mixed with prompts for classes that are present. Using samples from LVIS, Open Images and the Japanese Uncertainty Scenes (JUS) dataset, the authors compare detection quality (mAP), the ability to separate existing from missing labels (ROC/AUC), and confidence calibration.

GPT-4o performed worst. It localizes objects poorly, has the lowest mAP and outputs almost exclusively high confidence values, so it is strongly overconfident. The authors attribute its top AUC to the small number of predictions it makes. YOLO-World and Grounding DINO are mildly underconfident but far better calibrated. YOLO-World is strongest overall, while Grounding DINO hallucinates absent classes more often. Still, every model occasionally assigns high confidence to nonexistent objects, a real concern for high-stakes applications.

Key points

  • Models tested: YOLOv8l-world (via Ultralytics), groundingdino-base (via Hugging Face) and GPT-4o (via the OpenAI chat completion API, with a prompt that mimics the detectors' output format).
  • Datasets: validation data from LVIS (164,000 images, 1000 categories) and Open Images (about 9M images, 600 categories), plus JUS, which was built to test uncertainty in vision-language models. For LVIS, 2,000 samples were used, split evenly between existing and missing labels.
  • GPT-4o often ignored the format instructions. Valid responses came back for 89.2% (LVIS), 81.6% (Open Images) and 76.9% (JUS) of calls.
  • YOLO-World beat Grounding DINO on both mAP and AUC and more often declined to predict missing classes. Both detectors were slightly underconfident on LVIS and reasonably calibrated on Open Images.
  • Stated limitations:
    • The experiments used only subsets of the datasets.
    • Some "missing" classes were actually present but unlabeled and had to be removed by hand.
    • The analysis does not check whether the missing classes appeared in each model's training data.
    • The AUC figures rely only on true and false positives, so they should be interpreted cautiously.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 3The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksComputer Vision and Pattern Recognition · 2,869 works

Cite

Cite as Wout Teillers and Matias Valdenegro-Toro. 2026. Tricking Open-World Object Recognition Models: Uncertainty in Out-of-Distribution Detection. Proceedings of the 4th Workshop on Towards Knowledgeable Foundation Models (KnowFM 2026).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.knowfm-1.12

@inproceedings{teillers2026tricking,
  title = {{Tricking Open-World Object Recognition Models: Uncertainty in Out-of-Distribution Detection}},
  author = {Wout Teillers and Matias Valdenegro-Toro},
  booktitle = {Proceedings of the 4th Workshop on Towards Knowledgeable Foundation Models (KnowFM 2026)},
  year = {2026},
  doi = {10.18653/v1/2026.knowfm-1.12},
  url = {https://doi.org/10.18653/v1/2026.knowfm-1.12}
}

References

23 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Alain Andres, Aitor Martinez-Seras, Ibai Laña, and Javier Del Ser. 2024. On the black-box explainability of object detection models for safe and trustworthy industrial applications. Results in Engineering, 24:103498. find
  2. Weipeng Cao, Xuyang Yao, Zhiwu Xu, Ye Liu, Yinghui Pan, and Zhong Ming. 2025. A survey of zero-shot object detection. Big Data Mining and Analytics, 8(3):726–750. find
  3. Tianheng Cheng, Lin Song, Yixiao Ge, Wenyu Liu, Xinggang Wang, and Ying Shan. 2024. Yolo-world: Real-time open-vocabulary object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16901–16911. find
  4. Tom Fawcett. 2006. An introduction to roc analysis. Pattern recognition letters, 27(8):861–874. find
  5. Tobias Groot and Matias Valdenegro-Toro. 2024. Overconfidence is key: Verbalized uncertainty evaluation in large language and vision-language models. arXiv preprint arXiv:2405.02917. find
  6. Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger. 2017. On calibration of modern neural networks. In International conference on machine learning, pages 1321–1330. PMLR. find
  7. Agrim Gupta, Piotr Dollar, and Ross Girshick. 2019. Lvis: A dataset for large vocabulary instance segmentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 5356–5364. find
  8. Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qianglong Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, and 1 others. 2025. A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions. ACM Transactions on Information Systems,…
  9. Haibo Jin, Peiyan Zhang, Peiran Wang, Man Luo, and Haohan Wang. 2025. From hallucinations to jailbreaks: Rethinking the vulnerability of large foundation models. arXiv preprint arXiv:2505.24232. — [arXiv:2505.24232](https://arxiv.org/abs/2505.24232) find
  10. KJ Joseph, Salman Khan, Fahad Shahbaz Khan, and Vineeth N Balasubramanian. 2021. Towards open world object detection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 5830–5840. find
  11. Alina Kuznetsova, Hassan Rom, Neil Alldrin, Jasper Uijlings, Ivan Krasin, Jordi Pont-Tuset, Shahab Kamali, Stefan Popov, Matteo Malloci, Alexander Kolesnikov, Tom Duerig, and Vittorio Ferrari. 2020. The open images dataset v4: Unified image classification, object detection, and visual relationship d… find
  12. Yiming Li, Yi Wang, Wenqian Wang, Dan Lin, Bingbing Li, and Kim-Hui Yap. 2024. Open world object detection: A survey. IEEE Transactions on Circuits and Systems for Video Technology, 35(2):988–1008. find
  13. Tsung-Yi Lin, Licheng Ma, and Serge Belongie. 2014. Coco dataset: Detection evaluation. find
  14. Hanchao Liu, Wenyuan Xue, Yifei Chen, Dapeng Chen, Xiutian Zhao, Ke Wang, Liping Hou, Rongjun Li, and Wei Peng. 2024a. A survey on hallucination in large vision-language models. arXiv preprint arXiv:2402.00253. find
  15. Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang, Qing Jiang, Chunyuan Li, Jianwei Yang, Hang Su, and 1 others. 2024b. Grounding dino: Marrying dino with grounded pre-training for open-set object detection. In European Conference on Computer Vision, pages 38–55. Springer.
  16. Tianhe Ren, Qing Jiang, Shilong Liu, Zhaoyang Zeng, Wenlong Liu, Han Gao, Hongjie Huang, Zhengyu Ma, Xiaoke Jiang, Yihao Chen, and 1 others. 2024. Grounding dino 1.5: Advance the" edge" of open-set object detection. arXiv preprint arXiv:2405.10300. — [arXiv:2405.10300](https://arxiv.org/abs/2405.10300) find
  17. Amir Rosenfeld, Richard Zemel, and John K. Tsotsos. 2018. The elephant in the room. Preprint, arXiv:1808.03305. find
  18. Pranab Sahoo, Prabhash Meharia, Akash Ghosh, Sriparna Saha, Vinija Jain, and Aman Chadha. 2024. A comprehensive survey of hallucination in large language, image, video and audio foundation models. Findings of the Association for Computational Linguistics: EMNLP 2024, pages 11709–11724. find
  19. Matias Valdenegro-Toro. 2021. I find your lack of uncertainty in computer vision disturbing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1263–1272. find
  20. Jianwei Yang, Hao Zhang, Feng Li, Xueyan Zou, Chunyuan Li, and Jianfeng Gao. 2023. Set-of-mark prompting unleashes extraordinary visual grounding in gpt-4v. Preprint, arXiv:2310.11441. find
  21. Jingkang Yang, Kaiyang Zhou, Yixuan Li, and Ziwei Liu. 2024. Generalized out-of-distribution detection: A survey. International Journal of Computer Vision, 132(12):5635–5662. find
  22. Hao Zhang, Feng Li, Shilong Liu, Lei Zhang, Hang Su, Jun Zhu, Lionel M Ni, and Heung-Yeung Shum. 2022. Dino: Detr with improved denoising anchor boxes for end-to-end object detection. arXiv preprint arXiv:2203.03605. — [arXiv:2203.03605](https://arxiv.org/abs/2203.03605) find
  23. Zhengxia Zou, Keyan Chen, Zhenwei Shi, Yuhong Guo, and Jieping Ye. 2023. Object detection in 20 years: A survey. Proceedings of the IEEE, 111(3):257–276. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/teillers-2026-tricking-open-world-object-recognition/