Genetic network programming with reinforcement learning and optimal search component : a thesis presented in partial fulfilment of the requirements for the degree of Master of Science in Computer Sciences at Massey University, Auckland, New Zealand

Abstract

This thesis presents ways of improving the genetic composition, structure and learning strategies for a graph-based evolutionary algorithm, called Genetic Networking Programming with Reinforcement Learning (GNP-RL), particularly when working with multi-agent and dynamic environments. GNP-RL is an improvement over Genetic Programming, allowing for the concise representation of solutions in terms of a networked graph structure and uses RL to further refine the graph solutions. This work has improved GNP-RL by combining three new techniques: Firstly, it has added a reward and punishment scheme as part of its learning strategy that supports constraint conformance, allowing for a more adaptive training of the agent, so that it can learn how to avoid unwanted situations more effectively. Secondly, an optimal search algorithm has been combined in the GNP-RL core to get an accurate analysis of the exploratory environment. Thirdly, a task prioritization technique has been added to the agent’s learning by giving promotional rewards, so they are trained on how to take priority into account when performing tasks. In this thesis, we applied the improved algorithm to the Tile World benchmarking testbed, which is considered as one of the standard complex problems in this domain, having only a sparse training set. Our experiment results show that the proposed algorithm is superior than the best existing variant of the GNP-RL algorithm [1]. We have achieved 86.66% test accuracy on the standard benchmarking dataset [2]. In addition, we have created another benchmarking dataset, similar in complexity to the one proposed in [1], to test the proposed algorithms further, where it achieved a test accuracy of 96.66%; that is 33.66% more accurate.