Learning Swarm Behaviors using
Grammatical Evolution
and
Behavior Trees
Aadesh Neupane and Michael A. Goodrich
Problem
- Designing collective behaviors is cumbersome
- Requires experts for modeling
Solution
- Evolutionary computation to generate collective behaviors
$$\frac{dR}{dt} = -aR + vnD , R(0) = R_0$$
$$\frac{dO}{dt} = aR -bO +cE, O(0) = O_0$$
$$\frac{dE}{dt} = q(D)bO - cE, E(0) = E_0$$
$$\frac{dA}{dt} = p(D)bO - mA + wnD$$
$$\frac{dD}{dt} = mA - nD, D(0) = D_0$$
Reference: Stability of choice in the honey bee nest-site selection process
Presentation Structure
-
Related work
- Grammatical Evolution
- Behavior Trees
- Challenges
- GEESE-BT
- Fitness Functions
- Results
- Summary
Related Work - Evolutionary Robotics
Reference: Evolution of collective behaviors for a real swarm of aquatic surface robots
- Swarm behaviors like homing, dispersion, clustering and monitoring by Duarte et. at
- Validation of scalability, flexibility, and robustness on transferred controller
Neural Network based controllers
Merits
- Easy mapping from sensory inputs into actuators values
- Provides generalized solutions
- Only a few human inputs needed
Demerits
- Issues with reverse engineering
- Insights on collective behavior difficult
- Almost impossible to modify the behaviors
Related Work - Evolutionary Robotics
Reference: GESwarm: Grammatical Evolution for the Automatic Synthesis of Collective Behaviors in Swarm Robotics
- Foraging problem using GE by Ferrante et. al.
- Preconditions, low-level behavior and actions gave behaviors
Grammatical Evolution (GE) based controllers
Advantages
Disadvantages
- BNF grammar and objective function
- Analyzing and modifying collective behaviors easier
- Primitive low-level rules defined by experts
- Mapping to raw sensors and actuators values difficult
Related Work - Behavior Tree
Reference: Behavior Trees for Evolutionary Robotics
DelFly Explorer
Related Work - Evolution
Reference: odNEAT: An algorithm for distributed online, onboard evolution of robot behaviours
odNEAT
- Online distributed evolution of Neural networks
- Applicable for online learning in groups of embodied agents (robots)
- Performs well in aggregation task
Combination of best features from all these works?
- A multi-agent Grammatical Evolution with BT
GEESE
![](https://s3.amazonaws.com/media-p.slid.es/uploads/658435/images/4596860/robot_genome_2.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/658435/images/4596862/robot_genome.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/658435/images/4596864/robot_genome_1.png)
GEESE
Hello! Neighbour!
How are you doing with the phenotype in this environment?
Hello!
I collected 35 oz of water with the phenotype.
Take my genome and perform magic using genetic operators !
Agent
![](https://s3.amazonaws.com/media-p.slid.es/uploads/658435/images/5755675/agent.png)
Initialization
![](https://s3.amazonaws.com/media-p.slid.es/uploads/658435/images/5675760/initialization.png)
GEESE-BT Pipeline
![](https://s3.amazonaws.com/media-p.slid.es/uploads/658435/images/5763029/geese_archi.png)
Simulator
![](https://s3.amazonaws.com/media-p.slid.es/uploads/658435/images/5675762/system_diagram.png)
BNF Grammar - Swarms
![](https://s3.amazonaws.com/media-p.slid.es/uploads/658435/images/5755734/bnf_swarms.png)
Agent Behavior Chunks
![](https://s3.amazonaws.com/media-p.slid.es/uploads/658435/images/5675934/Composite_Drop.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/658435/images/5675937/Composite_Single_Carry.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/658435/images/5675939/Move_Towards.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/658435/images/5675941/Explore.png)
Composite Carry
Composite Drop
Move Towards
Explore
Challenges
Non-episodic learning
![](https://s3.amazonaws.com/media-p.slid.es/uploads/658435/images/5754494/po.jpg)
Partially observable
![](https://s3.amazonaws.com/media-p.slid.es/uploads/658435/images/5754517/20081229060928_Integrated_Behavior_Tree_Larger_System.png)
Huge search space
Credit Assignment
Solution - Fitness Functions
Task-Specific
- Foraging : Total food collection in hub
- Cooperative Transport : Total heavy object collected in hub
- Nest Maintenance : Total debris removed from hub
Spatial
- Exploration : Area explored
- Prospective : Total objects carried
Diversity
- Phenotypic Diversity : Total unique behaviors
Intrinsically Motivated
![](https://s3.amazonaws.com/media-p.slid.es/uploads/658435/images/6426371/fitness.png)
Solution - Fitness Functions
Single-Source Foraging
Results - Sampling
![](https://s3.amazonaws.com/media-p.slid.es/uploads/658435/images/6426393/sampling.png)
Results - Swarm Tasks
![](https://s3.amazonaws.com/media-p.slid.es/uploads/658435/images/6426394/all.png)
Results - BT
![](https://s3.amazonaws.com/media-p.slid.es/uploads/658435/images/6426395/btforge.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/658435/images/6426396/foraging_bt.png)
Evolved BT
Handcoded BT
Interesting Foraging Behavior
Interesting Nest Maintenance behavior
Future Work
-
General
- Test in actual robots
- Probabilistic modeling
-
Learning
- Solve regression and classification tasks
- General consensus decision-making
-
Swarms
- Interference effects
- Analysis of behavioral diversity and diversity heuristic
Acknowledgement
This work has been funded by ONR grant number N000141613025.
Thank You!
IJCAI
By Aadesh Neupane
IJCAI
IJCAI Presentation
- 789