Emergence of Collective Behaviors in Hub-Based Colonies using

Grammatical Evolution

and

Behavior Trees

Aadesh Neupane

Problem

- Designing collective behaviors is cumbersome

- Requires experts for modeling

Solution

- Evolutionary computation to generate collective behaviors

$$\frac{dR}{dt} = -aR + vnD , R(0) = R_0$$

$$\frac{dO}{dt} = aR -bO +cE, O(0) = O_0$$

$$\frac{dE}{dt} = q(D)bO - cE, E(0) = E_0$$

$$\frac{dA}{dt} = p(D)bO - mA + wnD$$

$$\frac{dD}{dt} = mA - nD, D(0) = D_0$$

Reference: Stability of choice in the honey bee nest-site selection process

Presentation Structure

Thesis Statement
Related Work
- Evolutionary Robotics
- Grammatical Evolution
- Behavior Trees
GEESE
- Algorithm
- BNF Grammar
- Objective Functions
Results
- Foraging
- Cooperative Transport
- Nest Maintenance

Thesis Statement

The interaction of hundreds of agents within the framework of distributed grammatical evolution will increase the effectiveness of evolving collective behaviors of bio-swarms. The evolved behaviors can be reused for different collective problems that have similar properties. Furthermore, with a slight variation of the objective function, the same set of primitive behaviors, encoded in the grammar, can lead to collective behaviors over a wider range of collective problems.

Related Work - Evolutionary Robotics

Reference: Evolution of collective behaviors for a real swarm of aquatic surface robots

Swarm behaviors like homing, dispersion, clustering and monitoring by Duarte et. at
Validation of scalability, flexibility, and robustness on transferred controller

Neural Network based controllers

Merits

Easy mapping from sensory inputs into actuators values
Provides generalized solutions
Only a few human inputs needed

Demerits

Issues with reverse engineering
Insights on collective behavior difficult
Almost impossible to modify the behaviors

Related Work - Evolutionary Robotics

Reference: Evolution of collective behaviors for a real swarm of aquatic surface robots

Neural Network based controllers

Related Work - Evolutionary Robotics

Reference: GESwarm: Grammatical Evolution for the Automatic Synthesis of Collective Behaviors in Swarm Robotics

Foraging problem using GE by Ferrante et. al.
Preconditions, low-level behavior and actions gave behaviors

Grammatical Evolution (GE) based controllers

Advantages

Disadvantages

BNF grammar and objective function
Analyzing and modifying collective behaviors easier

Primitive low-level rules defined by experts
Mapping to raw sensors and actuators values difficult

Related Work - Evolution

Reference: odNEAT: An algorithm for distributed online, onboard evolution of robot behaviours

odNEAT

Online distributed evolution of Neural networks
Applicable for online learning in groups of embodied agents (robots)
Performs well in aggregation task

Combination of best features from all these works?

- A multi-agent Grammatical Evolution

Related Work - Distributed Evolution

Reference: Behavior Trees for Evolutionary Robotics

DelFly Explorer

Related Work - Grammatical Evolution

BNF Grammar

Population of Genome

Corresponding Phenotype

Grammatical evolution: Evolving programs for an arbitrary language

Grammatical Evolution

BNF Grammar

<condition> ::= if_food_ahead(<progs>, <progs>)
<prog2>    ::= prog2(<progs>, <progs>)
<prog3>    ::= prog3(<progs>, <progs>, <progs>)
<op>    ::= left | right | move

- Santa Fe Trail

- Represented by tuple $N, T, P, S$

N -> Set of all non-terminals
T -> Set of all terminals
P -> Set of productions that map $N$ to $T$
S -> Initial start symbol

Grammatical Evolution

Santa Fe Trail

Objective
- Find all food using maximum of 600 moves
32 x 32 cells
Optimal trail
- 144 cells
- 89 food
- 55 gaps
Actions :
- Left
- Right
- Forward

Grammatical Evolution

Genome/ Genotype / Individual

Defines the proceedings of left-derivation
${Codon}$ is a group of symbols, usually 4 or 8.

Grammatical Evolution - BNF Grammar

Mapping

Let $c$ be codon integer
$A$ denotes the left-most non-terminal in the derivation
$r_A$ denotes the number of right-hand side rules associated with the production of $A$
RHSRule = $c\%r_A$

Phenotype

The output from the mapping process is the phenotype
The phenotype represents a valid expansion of the BNF grammar
$if\_food\_ahead(move, left)$

Example of Grammatical Evolution

RHSRule = $c\%r_A$

<code> ::= <code> | <progs>

<progs> ::= <condition> | <prog2> | <prog3> | <op>

<condition> ::= if_food_ahead(<progs>, <progs>)

<prog2> ::= prog2(<progs>, <progs>)

<prog3> ::= prog3(<progs>, <progs>, <progs>)

<op> ::= left | right | move

Grammatical Evolution

GE Pipeline

Related Work - Behavior Trees

Decision Tree
Sequential behavior composition
Subsumption Architecture

Behavior Trees

Sequence

Selector

Parallel

Decorator

Execution Node

Behavior Trees

Node	Succeeds	Fails	Running
Sequence	If all children succeed	If one child fails	If one child returns running
Selector	If one child succeeds	If all children fail	If one child returns running
Action	Task completion	Task impossible to complete	Task being computed
Condition	If true	If false	Never

BT Control Flow

Blackboard

Memory
Dictonary or Hash-table

GEESE

Hello! Neighbour!

How are you doing with the phenotype in this environment?

Hello!

I collected 35 oz of water with the phenotype.

Take my genome and perform magic using genetic operators !

Agent

Initialization

GEESE Pipeline

Framework

Swarm Behaviors

Composite Carry

Composite Drop

Move Towards

Explore

Fitness Functions

Task-Specific

Foraging : Total food collection in hub
Cooperative Transport : Total heavy object collected in hub
Nest Maintenance : Total debris removed from hub

Bootstrap

Exploration : Area explored
Prospective : Total objects carried

Diversity

Phenotypic Diversity : Total unique behaviors
Behavior Sampling : Transfer behaviors from learning to test environment

Example

Thesis Statement

Validation

Claim	Metric
Effectiveness	Quality solutions in fewer generations
Robustness	Same behavior for both single and multiple foraging problem
Breadth	Solutions for Foraging and cooperative transport problem

Validation

Effectiveness

Validation

Effectiveness

Solves the Santa Fe Trail in 324 steps

Validation

Robustness

Single Source Foraging problem

Multiple Source Foraging problem

Foraging

Hand-coded Behavior

Foraging

Evolved Behavior

Foraging

Evolved Behavior

Foraging Demo 1

Foraging Demo 2

Validation

- Breadth

Cooperative Transport

Nest Maintenance

Cooperative Transport

Hand-coded Behavior

Cooperative Transport

Evolved Behavior

Cooperative Transport

Evolved Behavior

Nest Maintanence

Hand-coded Behavior

Nest Maintanence

Evolved Behaviors

Nest Maintanence

Cooperative Transport Demo

Nest Maintenance Demo

Summary

Claims from thesis	Evidence
Effectiveness	Least number of steps for SantaFe Trail problem.
Robustness	Transferability of evolved behaviors in foraging problems
Breadth	Applicability of same BNF grammar for different swarm tasks
Novelty	Combination of GE with BT

Future Work

General
- Transfer the evolved behaviors to actual robots
- Probabilistic modeling of agent's sensing and acting capabilities
Learning
- Solve regression and classification type task
- Apply the algorithm for general consensus decision-making and NLP related problems
Swarms
- Interference effects with respect to agent size, obstacle density, and communication failures
- Modify the grammar to evolve behaviors for other swarms task
- Effects of behavioral diversity and diversity heuristic on quality and resilience of swarm behaviors

Acknowledgement

This work has been funded by ONR grant number N000141613025.

Thank You!

Master Thesis

By Aadesh Neupane