From Bayesian principles to Bayesian processes

This thesis considers the free energy principle (FEP) and its corollary, active inference, which form an explanatory framework that prescribes a Bayesian interpretation of self-organizing systems. The FEP originated in the domain of neuroscience, where it underwrote a unified theory that described perception, action and learning as emerging from minimizing a single objective function - variational free energy. However, since its conception, the FEP has transcended into physics and pure mathematics. Here, it presents itself as a set of mathematical arguments culminating in an inferential interpretation of a specific class of systems. The result has fundamentally changed the epistemological status of the FEP, moving it from the world of empirical hypotheses to the unfalsifiable territory of mathematical equivalences and tautological constructions. While the FEP may present a historical development that further unravels the symmetries that govern the laws of (our own) physics, its growth has left a range of epistemological confusion. In the current thesis, we evaluate how to maneuver from the principles of the FEP to the processes it purportedly explains. We identify four key areas in which the FEP can inform empirical science: 1) The FEP can aid us in designing intelligent agents by providing novel functionals that respect inherent uncertainty in the environment. We demonstrate equivalences between active inference and reinforcement learning, offer a novel implementation of active inference that utilizes amortized inference, and show that the proposed algorithm enables efficient exploration while offering improved sample efficiency compared to modern reinforcement learning algorithms. 2) We describe how the FEP can help us understand the nature of representation in living systems. Specifically, we show how the normative aspects of the FEP promote learning representations oriented towards action rather than veridical reconstructions of the environment. 3) We show how the FEP provides a framework for modeling perception, action, and learning in systems that can be empirically measured. An eye-tracking study demonstrates that an active inference model best explains human information-seeking, offering insights into the underlying mechanisms of perception and action. 4) In the final section, we ask whether active inference can inform the development of novel process theories in computational neuroscience. A biologically-plausible learning algorithm is developed and verified on various computer vision and reinforcement learning tasks. The resulting model explains a range of empirical phenomena and offers a new perspective on the role of bottom-up information in perception. This thesis affirms the role of the FEP and active inference as a generative framework for developing testable scientific theories.