University of Sussex
Browse
SUPN2 (8).pdf (8.54 MB)

Learning structured gaussians to approximate deep ensembles

Download (8.54 MB)
conference contribution
posted on 2023-06-10, 04:19 authored by Ivor SimpsonIvor Simpson, Sara Vicente, Neil D F Campbell
This paper proposes using a sparse-structured multivariate Gaussian to provide a closed-form approximator for the output of probabilistic ensemble models used for dense image prediction tasks. This is achieved through a convolutional neural network that predicts the mean and covariance of the distribution, where the inverse covariance is parameterised by a sparsely structured Cholesky matrix. Similarly to distillation approaches, our single network is trained to maximise the probability of samples from pretrained probabilistic models, in this work we use a fixed ensemble of networks. Once trained, our compact representation can be used to efficiently draw spatially correlated samples from the approximated output distribution. Importantly, this approach captures the uncertainty and structured correlations in the predictions explicitly in a formal distribution, rather than implicitly through sampling alone. This allows direct introspection of the model, enabling visualisation of the learned structure. Moreover, this formulation provides two further benefits: estimation of a sample probability, and the introduction of arbitrary spatial conditioning at test time. We demonstrate the merits of our approach on monocular depth estimation and show that the advantages of our approach are obtained with comparable quantitative performance.

History

Publication status

  • Published

File Version

  • Accepted version

Journal

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

ISSN

2575-7075

Publisher

IEEE/CVF

Page range

1-9

Event name

IEEE / CVF Computer Vision and Pattern Recognition Conference (CVPR)

Event location

New Orleans, Louisiana, USA

Event type

conference

Event date

19-24 June 2022

ISBN

9781665469463

Department affiliated with

  • Informatics Publications

Full text available

  • Yes

Peer reviewed?

  • Yes

Legacy Posted Date

2022-07-28

First Open Access (FOA) Date

2022-11-01

First Compliant Deposit (FCD) Date

2022-07-26

Usage metrics

    University of Sussex (Publications)

    Categories

    No categories selected

    Exports

    RefWorks
    BibTeX
    Ref. manager
    Endnote
    DataCite
    NLM
    DC