Learning Swarm Behaviors using

Grammatical Evolution

and

Behavior Trees

Aadesh Neupane and Michael A. Goodrich

Problem

- Designing collective behaviors is cumbersome

- Requires experts for modeling

Solution

- Evolutionary computation to generate collective behaviors

$$\frac{dR}{dt} = -aR + vnD , R(0) = R_0$$

$$\frac{dO}{dt} = aR -bO +cE, O(0) = O_0$$

$$\frac{dE}{dt} = q(D)bO - cE, E(0) = E_0$$

$$\frac{dA}{dt} = p(D)bO - mA + wnD$$

$$\frac{dD}{dt} = mA - nD, D(0) = D_0$$

Reference: Stability of choice in the honey bee nest-site selection process

Presentation Structure

  • Related work
    • Grammatical Evolution
    • Behavior Trees
  • Challenges
  • GEESE-BT
  • Fitness Functions
  • Results
  • Summary

Related Work - Evolutionary Robotics

Reference: Evolution of collective behaviors for a real swarm of aquatic surface robots

  • Swarm behaviors like homing, dispersion, clustering and monitoring by Duarte et. at
  • Validation of scalability, flexibility, and robustness on transferred controller

Neural Network based controllers

Merits

  • Easy mapping from sensory inputs into actuators values
  • Provides generalized solutions
  • Only a few human inputs needed

Demerits

  • Issues with reverse engineering
  • Insights on collective behavior difficult
  • Almost impossible to modify the behaviors

Related Work - Evolutionary Robotics

Reference: GESwarm: Grammatical Evolution for the Automatic Synthesis of Collective Behaviors in Swarm Robotics

  • Foraging problem using GE by Ferrante et. al.
  • Preconditions, low-level behavior and actions gave behaviors

Grammatical Evolution (GE) based controllers

Advantages

Disadvantages

  • BNF grammar and objective function
  • Analyzing and modifying collective behaviors easier
  • Primitive low-level rules defined by experts
  • Mapping to raw sensors and actuators values difficult

Related Work - Behavior Tree

Reference: Behavior Trees for Evolutionary Robotics

DelFly Explorer

Related Work - Evolution

Reference: odNEAT: An algorithm for distributed online, onboard evolution of robot behaviours

odNEAT

  • Online distributed evolution of Neural networks
  • Applicable for online learning in groups of embodied agents (robots)
  • Performs well in aggregation task

Combination of best features from all these works?

- A multi-agent Grammatical Evolution with BT

GEESE

GEESE

Hello! Neighbour!

How are you doing with the phenotype in this environment?

Hello!

I collected 35 oz of water with the phenotype.

Take my genome and perform magic using genetic operators !

Agent

Initialization

GEESE-BT Pipeline

Simulator

BNF Grammar - Swarms

Agent Behavior Chunks

Composite Carry

Composite Drop

Move Towards

Explore

Challenges

Non-episodic learning

Partially observable

Huge search space

Credit Assignment

Solution - Fitness Functions

Task-Specific

  • Foraging                             :   Total food collection in hub
  • Cooperative Transport   :   Total heavy object collected in hub
  • Nest Maintenance            :   Total debris removed from hub

Spatial

  • Exploration                         :   Area explored
  • Prospective                         :   Total objects carried

Diversity

  • Phenotypic Diversity        :    Total unique behaviors

Intrinsically Motivated

Solution - Fitness Functions

Single-Source Foraging

Results - Sampling

Results - Swarm Tasks

Results - BT

Evolved BT

Handcoded BT

Interesting Foraging Behavior

Interesting Nest Maintenance behavior

Future Work

  • General

    • Test in actual robots
    • Probabilistic modeling
  • Learning

    • Solve regression and classification  tasks
    • General consensus decision-making
  • Swarms

    • Interference effects
    • Analysis of behavioral diversity and diversity heuristic

Acknowledgement

This work has been funded by ONR grant number N000141613025.

Thank You!

IJCAI

By Aadesh Neupane