Learning Swarm Behaviors using
Grammatical Evolution
and
Behavior Trees
Aadesh Neupane and Michael A. Goodrich
Problem
- Designing collective behaviors is cumbersome
- Requires experts for modeling
Solution
- Evolutionary computation to generate collective behaviors
$$\frac{dR}{dt} = -aR + vnD , R(0) = R_0$$
$$\frac{dO}{dt} = aR -bO +cE, O(0) = O_0$$
$$\frac{dE}{dt} = q(D)bO - cE, E(0) = E_0$$
$$\frac{dA}{dt} = p(D)bO - mA + wnD$$
$$\frac{dD}{dt} = mA - nD, D(0) = D_0$$
Reference: Stability of choice in the honey bee nest-site selection process
Presentation Structure
-
Related work
- Grammatical Evolution
- Behavior Trees
- Challenges
- GEESE-BT
- Fitness Functions
- Results
- Summary
Related Work - Evolutionary Robotics
Reference: Evolution of collective behaviors for a real swarm of aquatic surface robots
- Swarm behaviors like homing, dispersion, clustering and monitoring by Duarte et. at
- Validation of scalability, flexibility, and robustness on transferred controller
Neural Network based controllers
Merits
- Easy mapping from sensory inputs into actuators values
- Provides generalized solutions
- Only a few human inputs needed
Demerits
- Issues with reverse engineering
- Insights on collective behavior difficult
- Almost impossible to modify the behaviors
Related Work - Evolutionary Robotics
Reference: GESwarm: Grammatical Evolution for the Automatic Synthesis of Collective Behaviors in Swarm Robotics
- Foraging problem using GE by Ferrante et. al.
- Preconditions, low-level behavior and actions gave behaviors
Grammatical Evolution (GE) based controllers
Advantages
Disadvantages
- BNF grammar and objective function
- Analyzing and modifying collective behaviors easier
- Primitive low-level rules defined by experts
- Mapping to raw sensors and actuators values difficult
Related Work - Behavior Tree
Reference: Behavior Trees for Evolutionary Robotics
DelFly Explorer
Related Work - Evolution
Reference: odNEAT: An algorithm for distributed online, onboard evolution of robot behaviours
odNEAT
- Online distributed evolution of Neural networks
- Applicable for online learning in groups of embodied agents (robots)
- Performs well in aggregation task
Combination of best features from all these works?
- A multi-agent Grammatical Evolution with BT
GEESE
GEESE
Hello! Neighbour!
How are you doing with the phenotype in this environment?
Hello!
I collected 35 oz of water with the phenotype.
Take my genome and perform magic using genetic operators !
Agent
Initialization
GEESE-BT Pipeline
Simulator
BNF Grammar - Swarms
Agent Behavior Chunks
Composite Carry
Composite Drop
Move Towards
Explore
Challenges
Non-episodic learning
Partially observable
Huge search space
Credit Assignment
Solution - Fitness Functions
Task-Specific
- Foraging : Total food collection in hub
- Cooperative Transport : Total heavy object collected in hub
- Nest Maintenance : Total debris removed from hub
Spatial
- Exploration : Area explored
- Prospective : Total objects carried
Diversity
- Phenotypic Diversity : Total unique behaviors
Intrinsically Motivated
Solution - Fitness Functions
Single-Source Foraging
Results - Sampling
Results - Swarm Tasks
Results - BT
Evolved BT
Handcoded BT
Interesting Foraging Behavior
Interesting Nest Maintenance behavior
Future Work
-
General
- Test in actual robots
- Probabilistic modeling
-
Learning
- Solve regression and classification tasks
- General consensus decision-making
-
Swarms
- Interference effects
- Analysis of behavioral diversity and diversity heuristic
Acknowledgement
This work has been funded by ONR grant number N000141613025.
Thank You!
IJCAI
By Aadesh Neupane
IJCAI
IJCAI Presentation
