Emergence of Collective Behaviors in Hub-Based Colonies using
Grammatical Evolution
and
Behavior Trees
Aadesh Neupane
Problem
- Designing collective behaviors is cumbersome
- Requires experts for modeling
Solution
- Evolutionary computation to generate collective behaviors
$$\frac{dR}{dt} = -aR + vnD , R(0) = R_0$$
$$\frac{dO}{dt} = aR -bO +cE, O(0) = O_0$$
$$\frac{dE}{dt} = q(D)bO - cE, E(0) = E_0$$
$$\frac{dA}{dt} = p(D)bO - mA + wnD$$
$$\frac{dD}{dt} = mA - nD, D(0) = D_0$$
Reference: Stability of choice in the honey bee nest-site selection process
Presentation Structure
- Thesis Statement
- Related Work
- Evolutionary Robotics
- Grammatical Evolution
- Behavior Trees
- GEESE
- Algorithm
- BNF Grammar
- Objective Functions
- Results
- Foraging
- Cooperative Transport
- Nest Maintenance
Thesis Statement
The interaction of hundreds of agents within the framework of distributed grammatical evolution will increase the effectiveness of evolving collective behaviors of bio-swarms. The evolved behaviors can be reused for different collective problems that have similar properties. Furthermore, with a slight variation of the objective function, the same set of primitive behaviors, encoded in the grammar, can lead to collective behaviors over a wider range of collective problems.
Related Work - Evolutionary Robotics
Reference: Evolution of collective behaviors for a real swarm of aquatic surface robots
- Swarm behaviors like homing, dispersion, clustering and monitoring by Duarte et. at
- Validation of scalability, flexibility, and robustness on transferred controller
Neural Network based controllers
Merits
- Easy mapping from sensory inputs into actuators values
- Provides generalized solutions
- Only a few human inputs needed
Demerits
- Issues with reverse engineering
- Insights on collective behavior difficult
- Almost impossible to modify the behaviors
Related Work - Evolutionary Robotics
Reference: Evolution of collective behaviors for a real swarm of aquatic surface robots
Neural Network based controllers
Related Work - Evolutionary Robotics
Reference: GESwarm: Grammatical Evolution for the Automatic Synthesis of Collective Behaviors in Swarm Robotics
- Foraging problem using GE by Ferrante et. al.
- Preconditions, low-level behavior and actions gave behaviors
Grammatical Evolution (GE) based controllers
Advantages
Disadvantages
- BNF grammar and objective function
- Analyzing and modifying collective behaviors easier
- Primitive low-level rules defined by experts
- Mapping to raw sensors and actuators values difficult
Related Work - Evolution
Reference: odNEAT: An algorithm for distributed online, onboard evolution of robot behaviours
odNEAT
- Online distributed evolution of Neural networks
- Applicable for online learning in groups of embodied agents (robots)
- Performs well in aggregation task
Combination of best features from all these works?
- A multi-agent Grammatical Evolution
Related Work - Distributed Evolution
Reference: Behavior Trees for Evolutionary Robotics
DelFly Explorer
Related Work - Grammatical Evolution
BNF Grammar
Population of Genome
Corresponding Phenotype
Grammatical evolution: Evolving programs for an arbitrary language
Grammatical Evolution
BNF Grammar
<code> ::= <code> | <progs>
<progs> ::= <condition> | <prog2> | <prog3> | <op>
<condition> ::= if_food_ahead(<progs>, <progs>)
<prog2> ::= prog2(<progs>, <progs>)
<prog3> ::= prog3(<progs>, <progs>, <progs>)
<op> ::= left | right | move
- Santa Fe Trail
- Represented by tuple \(N, T, P, S\)
- N -> Set of all non-terminals
- T -> Set of all terminals
- P -> Set of productions that map \(N\) to \(T\)
- S -> Initial start symbol
Grammatical Evolution
Santa Fe Trail
- Objective
- Find all food using maximum of 600 moves
- 32 x 32 cells
- Optimal trail
- 144 cells
- 89 food
- 55 gaps
- Actions :
- Left
- Right
- Forward
Grammatical Evolution
Genome/ Genotype / Individual
- Defines the proceedings of left-derivation
- \({Codon}\) is a group of symbols, usually 4 or 8.
Grammatical Evolution - BNF Grammar
Mapping
- Let \(c\) be codon integer
- \(A\) denotes the left-most non-terminal in the derivation
- \(r_A\) denotes the number of right-hand side rules associated with the production of \(A\)
-
RHSRule = \(c\%r_A\)
Phenotype
- The output from the mapping process is the phenotype
- The phenotype represents a valid expansion of the BNF grammar
- \(if\_food\_ahead(move, left)\)
Example of Grammatical Evolution
RHSRule = \(c\%r_A\)
<code> ::= <code> | <progs>
<progs> ::= <condition> | <prog2> | <prog3> | <op>
<condition> ::= if_food_ahead(<progs>, <progs>)
<prog2> ::= prog2(<progs>, <progs>)
<prog3> ::= prog3(<progs>, <progs>, <progs>)
<op> ::= left | right | move
Grammatical Evolution
GE Pipeline
Related Work - Behavior Trees
- Decision Tree
- Sequential behavior composition
- Subsumption Architecture
Behavior Trees
Sequence
Selector
Parallel
Decorator
Execution Node
Behavior Trees
| Node | Succeeds | Fails | Running |
|---|---|---|---|
| Sequence | If all children succeed | If one child fails | If one child returns running |
| Selector | If one child succeeds | If all children fail | If one child returns running |
| Action | Task completion | Task impossible to complete | Task being computed |
| Condition | If true | If false | Never |
BT Control Flow
Blackboard
- Memory
- Dictonary or Hash-table
GEESE
GEESE
Hello! Neighbour!
How are you doing with the phenotype in this environment?
Hello!
I collected 35 oz of water with the phenotype.
Take my genome and perform magic using genetic operators !
Agent
Initialization
GEESE Pipeline
Framework
Swarm Behaviors
Swarm Behaviors
Composite Carry
Composite Drop
Move Towards
Explore
Fitness Functions
Task-Specific
- Foraging : Total food collection in hub
- Cooperative Transport : Total heavy object collected in hub
- Nest Maintenance : Total debris removed from hub
Bootstrap
- Exploration : Area explored
- Prospective : Total objects carried
Diversity
- Phenotypic Diversity : Total unique behaviors
- Behavior Sampling : Transfer behaviors from learning to test environment
Example
Thesis Statement
The interaction of hundreds of agents within the framework of distributed grammatical evolution will increase the effectiveness of evolving collective behaviors of bio-swarms. The evolved behaviors can be reused for different collective problems that have similar properties. Furthermore, with a slight variation of the objective function, the same set of primitive behaviors, encoded in the grammar, can lead to collective behaviors over a wider range of collective problems.
Validation
| Claim | Metric |
|---|---|
| Effectiveness | Quality solutions in fewer generations |
| Robustness | Same behavior for both single and multiple foraging problem |
| Breadth |
Solutions for Foraging and cooperative transport problem |
Validation
Effectiveness
Validation
Effectiveness
Solves the Santa Fe Trail in 324 steps
Validation
Robustness
Single Source Foraging problem
Multiple Source Foraging problem
Foraging
Foraging
Hand-coded Behavior
Foraging
Evolved Behavior
Foraging
Evolved Behavior
Foraging Demo 1
Foraging Demo 2
Validation
- Breadth
Cooperative Transport
Nest Maintenance
Cooperative Transport
Cooperative Transport
Hand-coded Behavior
Cooperative Transport
Evolved Behavior
Cooperative Transport
Evolved Behavior
Nest Maintanence
Nest Maintanence
Hand-coded Behavior
Nest Maintanence
Evolved Behaviors
Nest Maintanence
Cooperative Transport Demo
Nest Maintenance Demo
Summary
| Claims from thesis | Evidence |
|---|---|
| Effectiveness | Least number of steps for SantaFe Trail problem. |
| Robustness | Transferability of evolved behaviors in foraging problems |
| Breadth | Applicability of same BNF grammar for different swarm tasks |
| Novelty | Combination of GE with BT |
Future Work
-
General
- Transfer the evolved behaviors to actual robots
- Probabilistic modeling of agent's sensing and acting capabilities
-
Learning
- Solve regression and classification type task
- Apply the algorithm for general consensus decision-making and NLP related problems
-
Swarms
- Interference effects with respect to agent size, obstacle density, and communication failures
- Modify the grammar to evolve behaviors for other swarms task
- Effects of behavioral diversity and diversity heuristic on quality and resilience of swarm behaviors
Acknowledgement
This work has been funded by ONR grant number N000141613025.