icjnn.pdf (406 kB)
Scaling active inference
conference contribution
posted on 2023-06-09, 21:16 authored by Alexander Tschantz, Manuel Baltieri, Anil SethAnil Seth, Christopher BuckleyChristopher BuckleyIn reinforcement learning (RL), agents often operate in partially observed and uncertain environments. Model-based RL suggests that this is best achieved by learning and exploiting a probabilistic model of the world. ‘Active inference’ is an emerging normative framework in cognitive and computational neuroscience that offers a unifying account of how biological agents achieve this. On this framework, inference, learning and action emerge from a single imperative to maximize the Bayesian evidence for a niched model of the world. However, implementations of this process have thus far been restricted to low-dimensional and idealized situations. Here, we present a working implementation of active inference that applies to high-dimensional tasks, with proof-of-principle results demonstrating efficient exploration and an order of magnitude increase in sample efficiency over strong model-free baselines. Our results demonstrate the feasibility of applying active inference at scale and highlight the operational homologies between active inference and current model-based approaches to RL.
Funding
The Sackler Centre for Consciousness Science 2019-2021 Leading-edge consciousness science and its application to psychological and neurological health; G2608; SACKLER-DR MORTIMER AND THERESA SACKLER FOUNDATION
Sackler Centre - donation; G1813; SACKLER-DR MORTIMER AND THERESA SACKLER FOUNDATION
History
Publication status
- Published
File Version
- Accepted version
Journal
2020 International Joint Conference on Neural Networks (IJCNN)ISSN
2161-4393Publisher
IEEEExternal DOI
Event name
IEEE World Congress On Computational Intelligence: The International Joint Conference on Neural Networks (IJCNN)Event location
GlasgowEvent type
conferenceEvent date
July 19 - 24th 2020ISBN
9781728169279Department affiliated with
- Informatics Publications
Notes
© 2020 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other worksFull text available
- Yes
Peer reviewed?
- Yes
Legacy Posted Date
2020-06-15First Open Access (FOA) Date
2020-09-29First Compliant Deposit (FCD) Date
2020-06-09Usage metrics
Categories
No categories selectedKeywords
Licence
Exports
RefWorks
BibTeX
Ref. manager
Endnote
DataCite
NLM
DC