University of Sussex
Browse
icjnn.pdf (406 kB)

Scaling active inference

Download (406 kB)
conference contribution
posted on 2023-06-09, 21:16 authored by Alexander Tschantz, Manuel Baltieri, Anil SethAnil Seth, Christopher BuckleyChristopher Buckley
In reinforcement learning (RL), agents often operate in partially observed and uncertain environments. Model-based RL suggests that this is best achieved by learning and exploiting a probabilistic model of the world. ‘Active inference’ is an emerging normative framework in cognitive and computational neuroscience that offers a unifying account of how biological agents achieve this. On this framework, inference, learning and action emerge from a single imperative to maximize the Bayesian evidence for a niched model of the world. However, implementations of this process have thus far been restricted to low-dimensional and idealized situations. Here, we present a working implementation of active inference that applies to high-dimensional tasks, with proof-of-principle results demonstrating efficient exploration and an order of magnitude increase in sample efficiency over strong model-free baselines. Our results demonstrate the feasibility of applying active inference at scale and highlight the operational homologies between active inference and current model-based approaches to RL.

Funding

The Sackler Centre for Consciousness Science 2019-2021 Leading-edge consciousness science and its application to psychological and neurological health; G2608; SACKLER-DR MORTIMER AND THERESA SACKLER FOUNDATION

Sackler Centre - donation; G1813; SACKLER-DR MORTIMER AND THERESA SACKLER FOUNDATION

History

Publication status

  • Published

File Version

  • Accepted version

Journal

2020 International Joint Conference on Neural Networks (IJCNN)

ISSN

2161-4393

Publisher

IEEE

Event name

IEEE World Congress On Computational Intelligence: The International Joint Conference on Neural Networks (IJCNN)

Event location

Glasgow

Event type

conference

Event date

July 19 - 24th 2020

ISBN

9781728169279

Department affiliated with

  • Informatics Publications

Notes

© 2020 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works

Full text available

  • Yes

Peer reviewed?

  • Yes

Legacy Posted Date

2020-06-15

First Open Access (FOA) Date

2020-09-29

First Compliant Deposit (FCD) Date

2020-06-09

Usage metrics

    University of Sussex (Publications)

    Categories

    No categories selected

    Exports

    RefWorks
    BibTeX
    Ref. manager
    Endnote
    DataCite
    NLM
    DC