ENERGY-BASED OUT-OF-DISTRIBUTION DETECTION

Dao Minh Tuan; Do Cong To; Nguyen Mau Truong Giang

Dao Minh Tuan Hung Yen University of Technology and Education
Do Cong To Vietnam Electricity
Nguyen Mau Truong Giang Vietnam Posts and Telecommunications Group - AI

Keywords: out-of-distribution detection, out-of-distribution detection model, Energy-based machine learning model, Machine learning

Abstract

Energy-Based Models (EBM) are attracting particular interest in the field of machine learning and artificial intelligence due to their ability to detect abnormal distributions. The importance of EBM lies in its wide applicability in many practical fields, from cybersecurity, industrial production monitoring to healthcare. In this paper, we provide a deeper look into the use of EBM for abnormal distribution detection. The article presents the basic theory of EBM, compares it with other models, and conducts experiments on the CIFAR-10 and CIFAR-100 image datasets. The results of the study indicate that EBM has potential in detecting abnormal distributions and presents challenges and potential directions for future development. This paper expands knowledge of EBM and provides a basis for potential real-world applications.

References

Petra Bevandi´c, Ivan Krešo, Marin Orši´c, and Siniša Šegvi´c. Discriminative out-of-distribution detection for semantic segmentation. arXiv preprint arXiv:1808.07703, 2018.

Jiefeng Chen, Yixuan Li, Xi Wu, Yingyu Liang, and Somesh Jha. Informative outlier matters: Robustifying out-of-distribution detection using outlier mining. arXiv preprint arXiv:2006.15207, 2020.

Dan Hendrycks and Kevin Gimpel. A baseline for detecting misclassified and out-of-distribution examples in neural networks. arXiv preprint arXiv:1610.02136, 2016.

Dan Hendrycks, Mantas Mazeika, and Thomas Dietterich. Deep anomaly detection with outlier exposure. In International Conference on Learning Representations, 2019.

Yen-Chang Hsu, Yilin Shen, Hongxia Jin, and Zsolt Kira. Generalized ODIN: Detecting out-of--distribution image without learning from out-of-distribution data. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 10951-10960.

Rui Huang and Yixuan Li. Mos: Towards scaling out-of-distribution detection for large semantic space. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2021.

Ziqian Lin, Sreya Dutta, and Yixuan Li. Mood: Multi-level out-of-distribution detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2021.

Yann LeCun, Sumit Chopra, Raia Hadsell, Marc’Aurelio Ranzato, and Fu-Jie Huang. A tutorial on energy-based learning. In G. Bakir, T. Hofman, B. Schölkopf, A. Smola, and B. Taskar, editors, Predicting Structured Data. MIT Press, 2006.

Weitang Liu, Xiaoyun Wang, John D. Owens, and Yixuan Li. Energy-based Out-of-distribution Detection, 34th Conference on Neural Information Processing Systems (NeurIPS 2020), Vancouver, Canada, 2020.

Eric Nalisnick, Akihiro Matsukawa, Yee Whye Teh, Dilan Gorur, and Balaji Lakshminarayanan. Do deep generative models know what they don’t know? arXiv preprint arXiv:1810.09136, 2018.

Jinsong Zhang, Qiang Fu, Xu Chen, Lun Du, Zelin Li, Gang Wang, xiaoguang Liu, Shi Han, Dongmei Zhang, Out-of-Distribution Detection based on In-Distribution Data Patterns Memorization with Modern Hopfield Energy, ICLR 2023 Conference Program Chairs, 2023.

Junbo Zhao, Michael Mathieu and Yann LeCun, Energy-Based Generative Adversarial Networks, conference paper at ICLR 2017.

Takayuki Osogami, Boltzmann machines and energy-based models, Neural and Evolutionary Computing, 2019.