isal_a_00196.pdf (1.94 MB)
Evolving recurrent neural network controllers by incremental fitness shaping
Version 2 2023-06-12, 09:07
Version 1 2023-06-09, 17:59
conference contribution
posted on 2023-06-12, 09:07 authored by Kaan Akinci, Andy PhilippidesAndy PhilippidesTime varying artificial neural networks are commonly used for dynamic problems such as games controllers and robotics as they give the controller a memory of what occurred in previous states which is important as actions in previous states can influence the final success of the agent. Because of this temporal dependence, methods such as back-propagation can be difficult to use to optimise network parameters and so genetic algorithms (GAs) are often used instead. While recurrent neural networks (RNNs) are a common network used with GAs, long short term memory (LSTM) networks have had less attention. Since, LSTM networks have a wide range of temporal dynamics, in this paper, we evolve an LSTM network as a controller for a lunar lander task with two evolutionary algorithms: a steady state GA (SSGA) and an evolutionary strategy (ES). Due to the presence of a large local optima in the fitness space, we implemented an incremental fitness scheme to both evolutionary algorithms. We also compare the behaviour and evolutionary progress of the LSTM with the behaviour of an RNN evolved via NEAT and ES with the same fitness function. LSTMs proved themselves to be evolvable on such tasks, though the SSGA solution was outperformed by the RNN. However, despite using an incremental scheme, the ES developed solutions far better than both showing that ES can be used both for incremental fitness and for LSTMs and RNNs on dynamic tasks.
History
Publication status
- Published
File Version
- Published version
Journal
Proceedings of the ALIFE Conference 2019 (ALIFE 2019): A Hybrid of the European Conference on Artificial Life (ECAL) and the International Conference on the Synthesis and Simulation of Living Systems (ALIFE)Publisher
The MIT PressExternal DOI
Page range
416-423Event name
ALIFE 2019Event location
Newcastle upon Tyne, UKEvent type
conferenceEvent date
29 July - 2nd August, 2019Department affiliated with
- Informatics Publications
Research groups affiliated with
- Centre for Computational Neuroscience and Robotics Publications
Full text available
- Yes
Peer reviewed?
- Yes
Editors
Harold Fellermann, Angel Goñi-Moreno, Jaume Bacardit, Rudolf Marcel FüchslinLegacy Posted Date
2019-06-05First Open Access (FOA) Date
2019-06-07First Compliant Deposit (FCD) Date
2019-06-04Usage metrics
Categories
No categories selectedKeywords
Licence
Exports
RefWorks
BibTeX
Ref. manager
Endnote
DataCite
NLM
DC