Laplace Approximation with Diagonalized Hessian for Over-parameterized Neural Networks

Ming Gui, Ziqing Zhao, Tianming Qiu und Hao Shen

NeurIPS 2021, Bayesian Deep Learning Workshop(36)

Dezember 2021


Bayesian Neural Networks (BNNs) provide valid uncertainty estimation on their feedforward outputs. However, it can become computationally prohibitive to apply them to modern large-scale neural networks. In this work, we combine the Laplace approximation with linearized inference for a real-time and robust uncertainty evaluation. Specifically, we study the effectiveness and computational necessity of a diagonal Hessian approximation in the Laplace approximation on over-parameterized networks. The proposed approach is investigated on object detection tasks in an autonomous driving scenario and demonstrates faster inference speed and convincing results.