Interactive Evolutionary Computation

Tim Thornton

Presentation Overview

 

  1. What is IEC
  2. Motivation
  3. Overview of Field
  4. Technical Aspects
  5. Problems...
  6. Solutions?
  7. Towards a Theory of IGA
  8. Applications and Results of IEC
  9. Conclusion

What is IEC?

Human input can also be benficial in             

  • Evolution Strategies (IES)
  • Genetic Programming (IGP)
  • Human-Based Genetic Algorithms (HBGA)
  • Interactive Swarm Optimization (ISO)

"Human in the loop" is not restricted only to interactive GA (IGA)

Interactive Evolutionary Computation

Human user is involved in evolutionary computation in some way

IGA vs HBGA

IGA has minimum necessary human involvement

  • Fitness/selection

HBGA has human involvement at every step

  • Fitness/selection
  • Initialization of Population
  • Recombinant Crossover
  • Mutation

Interactive Evolutionary Computation

What is IEC?

Interactive Evolutionary Computation

Two definitions:

Narrow -

User aids fitness function

(IGA)

Broad -

User aids entire search

(HBGA)

 

Selection
Recombination		 Computational Human
Computational SGA/PGA IGA
Human CAD HBGA

 

Presentation Overview

 

  1. What is IEC
  2. Motivation
  3. Overview of Field
  4. Technical Aspects
  5. Problems...
  6. Solutions?
  7. Towards a Theory of IGA
  8. Applications and Results of IEC
  9. Conclusion

Motivation

We know how to do this with

non-interactive GA

http://rogeralsing.com/2008/12/07/genetic-programming-evolution-of-mona-lisa/

Motivation

What if we don't have the target?

Maybe we're not

great artists...

Aoki, K. and Tagaki, H.: 3D CG Lighting with Interactive GA

Motivation

IEC can assist in human creativity

Used in many creative domains:

  • Visual art
    • 3D lighting
    • 3D/2D image creation 
  • Music
    • Melody generation
    • Rhythm Generation
    • Synthesizer optimization
  • Industrial design 
  • etc

Beyond creative domains, IEC is useful in any case where the fitness is difficult to formalize

Presentation Overview

 

  1. What is IEC
  2. Motivation
  3. Overview of Field
  4. Technical Aspects
  5. Problems...
  6. Solutions?
  7. Towards a Theory of IGA
  8. Applications and Results of IEC
  9. Conclusion

Overview of Field

  • First introduced in 1986 with the biomorphs of Dawkins' The Blind Watchmaker
    • ​Interactive L-Systems 
  • Early/mid 1990's saw 
    applications in graphic art and CG
  • 1995 - number of papers jumped 
    from 11 to 23
    • Appl. in data mining and music introduced
  • 1997 - number of papers jumped from 22 to 48
    • New interest in research to reduce fatigue
  • By 2000, > 250 papers published

Overview of Field

Most papers in

  • Graphic art and CG animation
  • Music
  • Industrial design
  • Facial image recognition
  • Database retrieval
  • Data Mining

New applications for consumers

  • Make your own fish at virtual aquarium in Japan
  • Make your own art at multimedia museum in Tokyo

Presentation Overview

 

  1. What is IEC
  2. Motivation
  3. Overview of Field
  4. Technical Aspects
  5. Problems...
  6. Solutions?
  7. Towards a Theory of IGA
  8. Applications and Results of IEC
  9. Conclusion

Technical Aspects

    Procedure IEA() { 
        t = 0; 
        Initialize Population(t); 
        PresentPopulationToUser(t); 
        while (Not Done) { 
            Parents (t) = UserSelectParents(Population (t)); 
            Offspring (t) = Recombine(Parents (t)); 
            Offspring (t) = Mutate(Offspring (t)); 
            PresentPopulationToUser(Offspring (t)); 
            Population (t+1) = UserSelectSurvivors(Population (t), Offspring (t)); 
            t = t + 1; 
        }
    }  
    Procedure EA() {   
        t = 0;   
        Initialize Population(t);   
        Evaluate Population(t);   
        while (Not Done) {    
            Parents (t)   = SelectParents (Population(t));    
            Offspring (t) = Recombine (Parents(t));    
            Offspring (t) = Mutate (Offspring(t));    
            Evaluate (Offspring(t));    
            Population(t+1) = SelectSurvivors(Population(t), Offspring(t));    
            t = t + 1;   
        } 
    }

Genetic Algorithm

Interactive Genetic Algorithm

Technical Aspects

    Procedure IEA() { 
        t = 0; 
        Initialize Population(t); 
->      PresentPopulationToUser(t); 
        while (Not Done) { 
->          Parents (t) = UserSelectParents(Population (t)); 
            Offspring (t) = Recombine(Parents (t)); 
            Offspring (t) = Mutate(Offspring (t)); 
->          PresentPopulationToUser(Offspring (t)); 
->          Population (t+1) = UserSelectSurvivors(Population (t), Offspring (t)); 
            t = t + 1; 
        }
    }  
    Procedure EA() {   
        t = 0;   
        Initialize Population(t);   
        Evaluate Population(t);   
        while (Not Done) {    
            Parents (t)   = SelectParents (Population(t));    
            Offspring (t) = Recombine (Parents(t));    
            Offspring (t) = Mutate (Offspring(t));    
            Evaluate (Offspring(t));    
            Population(t+1) = SelectSurvivors(Population(t), Offspring(t));    
            t = t + 1;   
        } 
    }

Genetic Algorithm

Interactive Genetic Algorithm

Technical Aspects

  • IEC uses two different spaces for search
    • Human evaluates output according to psychological space
    • EC searches in feature parameter space
  • IEC is an optimization technology where EC and human search are cooperatively based on the mapping between the two spaces

Takagi: Interactive Evolutionary Computation: System Optimization Based on Human Subjective Evaluation

Technical Aspects

  1. Humans evaluate system output rather than phenotype 
  2. Evaluations tend to fluctuate
    • It is reported that convergence is not practically influenced by fluctuation
      • IEC converges to an area rather than an exact point (targetting impression area)
  3. Evaluations are relative rather than absolute

Points to consider:

Technical Aspects

  • Still optimization technology
    • Human evaluator applies fitness to individuals found during a search through a parameter space
    • Relevant parameters need to be known
    • This is what separates IEC from pure design

IEC vs Design

Presentation Overview

 

  1. What is IEC
  2. Motivation
  3. Overview of Field
  4. Technical Aspects
  5. Problems...
  6. Solutions?
  7. Towards a Theory of IGA
  8. Applications and Results of IEC
  9. Conclusion

Problems...

  • "The Fitness Bottleneck"
  • Human Evaluator means:
    • Small population, few generations
    • Likely need to construct new generations and system output in near real time
  • Need EA that can converge under these contraints
    • ~10-20 generations
    • Population determined by complexity of individuals:
      • Eg. Number of images that fit on screen
      • Number of songs/movies that fit in human memory

Problems...

The big two problems:

 

  1. How to reduce human fatigue?
  2. How to increase convergence and accuracy while using small populations and generations

Presentation Overview

 

  1. What is IEC
  2. Motivation
  3. Overview of Field
  4. Technical Aspects
  5. Problems...
  6. Solutions?
  7. Towards a Theory of IGA
  8. Applications and Results of IEC
  9. Conclusion

Solutions?

Discrete Fitness Value Input

  • Qualitatively evaluating individuals causes psychological stress
  • Using small rough rating of individuals reduces burden
  • Found "continuous" evaluation to be better after 100th generation
  • "Discrete" input has similar average fitness under 100 generations
  • IEC is only practical to ~20 generations
  • Most applications use either binary or 5-point scale

Solutions?

Visualized IEC

  • Hayashida, Takagi - Kyushu Institute of Design
  • Goal: reduce human fatigue
  • Idea:
    •  Visualize nD individuals in 2D space
    • Allow user intervention in search using 2D mapping
  • Hypothesis:
    • Active user intervention in search leads to faster convergence
  • Why?
    • Combine capabilities of human and EC to search for global optimum

Solutions?

Visualized IEC

  • Human involved in:
    • Fitness evaluation
    • Selection of 2D individual(s) for use in next generation
  • The worst individual in the parent pop is replaced by 2D selection(s)
  • Crossover/mutation occur as usual by EC

Hayashida, N., Takagi, H.: Visualized IEC: Interactive Evolutionary Computation with Multidimensional Data Visualization

Solutions?

Visualized IEC

  • Many potential mapping techniques from nD to 2D
    • Principle component analysis
    • Sammon's nonlinear maps
    • Visor
    • TOPAS
  • Information is lost when reducing dimensions
    • Mapping technique needs to somewhat maintain topological relationships of individuals

Hayashida, N., Takagi, H.: Visualized IEC: Interactive Evolutionary Computation with Multidimensional Data Visualization

Solutions?

Visualized IEC

VIEC Evaluation:

  • Self-organizing maps
  • Regular GA used to find min coordinate of Schaffer's second function in 3 and 5 dimensions
    •  
  • Fitness function is distance of coordinate 
    • Human only used to select 2D individuals between generations
  • Population size = 20, Generations = 10
  • Result: VGA (p=20) similar to both                                                                     GA (p=100) and GA (p=1000)
  • VGA ~ 5x faster convergence
(\sum_{i=1}^n x_i^2)^{1 \over 4} * [sin^2 (50 * (\sum_{i=1}^mx_i^2)^{1\over 10} + 1)], -100 \le x_i \le 100
(∑​i=1​n​​x​i​2​​)​​4​​1​​​​∗[sin​2​​(50∗(∑​i=1​m​​x​i​2​​)​​10​​1​​​​+1)],−100≤x​i​​≤100

Hayashida, N., Takagi, H.: Visualized IEC: Interactive Evolutionary Computation with Multidimensional Data Visualization

Solutions?

Visualized IEC : Speech Synthesis

Hayashida, N., Takagi, H.: Visualized IEC: Interactive Evolutionary Computation with Multidimensional Data Visualization

Solutions?

On-line knowledge embedding

  • Takagi, Kishi
  • Kyushu Institue of Design
  • Goal: reduce search space with search hints, thereby accelerating convergence

Solutions?

On-line knowledge embedding

OLKE Evaluation:

  • Facial montage system
  • Ability to freeze part of the face once satisfactory
  • Search space is 30 student faces with 6 parts each, 
  • Compared to non OLKE system
  • Found OLKE system to converge faster with more accuracy

Takagi, H. Kishi, K.: On-line Knowledge Embedding for an Interactive EC-based Montage System

Solutions?

Chromosome Appearance Probability Matrix

  • Idea: Extend GA with oriented mutation for faster convergence
  • Faster convergence leads to less human fatigue
  • An nD array M used to store combinations of alleles
  •  
  •  
  •  
  • Array shows probability that each combination of alleles has of being chosen
  • One or two individuals selected each iteration
    • Corresponding position in array is updated
    •  
M(gene_1, ...,gene_n) = {1 \over T}
M(gene​1​​,...,gene​n​​)=​T​​1​​
T=\prod_{i=1}^nm_i
T=∏​i=1​n​​m​i​​
gene_i = [allele_1,...,allele_m]
gene​i​​=[allele​1​​,...,allele​m​​]
\Delta M=[M_{gene_s^1,...,gene_s^n} \times (1.0 + \alpha)] - M_{gene_s^1,...,gene_s^n}
ΔM=[M​gene​s​1​​,...,gene​s​n​​​​×(1.0+α)]−M​gene​s​1​​,...,gene​s​n​​​​

Solutions?

Chromosome Appearance Probability Matrix

    procedure CAPM {
        t <- 0
        Initialize (Pa)
        InitProbMatrix(Mp)
        Evaluate(Pa)
        while (Not Done) {
            Parents <- SelectParents(Pt)
            UpdateProbMatrix(Mt, a)
            Offspring <- Crossover(Parents)
            OrientedMutation(Offspring)
            Evaluate(Offspring)
            t <- t + 1
        }
    }

CAPM Algorithm

Need to add 2 things:

  1. Training of appearance matrix
  2. Oriented mutation using trained matrix

Solutions?

Chromosome Appearance Probability Matrix

 

G1

G2

 1 2 3 4
0 0.045 0.025 0.025 0.02
1 0.015 0.025 0.025 0.015
2 0.025 0.01 0.025 0.025
3 0.025 0.04 0.025 0.04
4 0.025 0.025 0.025 0.025
5 0.025 0.025 0.025 0.025
6 0.015 0.025 0.02 0.025
7 0.025 0.025 0.02 0.025
8 0.025 0.025 0.035 0.025
9 0.025 0.025 0.025 0.025

 

\Delta m = [M[2,3] \times (1 + \alpha)] - M[2,3]
Δm=[M[2,3]×(1+α)]−M[2,3]
\Delta m = [0.04 \times (1 + 0.005)] - 0.04]
Δm=[0.04×(1+0.005)]−0.04]
\Delta m = 0.002
Δm=0.002
M[2,3] = M[2,3] + \Delta m = 0.0402
M[2,3]=M[2,3]+Δm=0.0402
M[1,0] = M[1,0] - {\Delta m \over 40} = 0.044995
M[1,0]=M[1,0]−​40​​Δm​​=0.044995
\vdots
⋮
M[4,9] = M[4,9] - {\Delta m \over 40} = 0.024995
M[4,9]=M[4,9]−​40​​Δm​​=0.024995

Probability Matrix

Update Procedure

Solutions?

Chromosome Appearance Probability Matrix

When individual I selected for mutation:

  1. Reference chromosome R generated by sampling M
  2. Randomly choose position in I to mutate
  3. Replace chosen gene from I with corresponding gene from R

Mutation Procedure

Solutions?

Chromosome Appearance Probability Matrix

  • Evaluation: Rastrigin in 2D
  • CAPM(p=10) ~ GA(p=100) after 100 generations

Presentation Overview

 

  1. What is IEC
  2. Motivation
  3. Overview of Field
  4. Technical Aspects
  5. Problems...
  6. Solutions?
  7. Towards a Theory of IGA
  8. Applications and Results of IEC
  9. Conclusion

Towards a Theory

One major attempt to create theoretical model of IEC

Günter Rudolph:

  • Can IEC be modeled in a probabilistic framework?
  • If so, is there any utility?

Towards a Theory

Classic EAs can be modeled by Markov Chains

Too restricted for IEA

Rudolph attempts to model IEA using stochastic automata

Towards a Theory

Stochastic Automata

\langle{S, X, Y, P(s', y|s, x)}\rangle
⟨S,X,Y,P(s​′​​,y∣s,x)⟩
S=States
S=States
Y=Output\ Symbols
Y=Output Symbols
X = Input\ Symbols
X=Input Symbols
P:S \times Y \times S \times X \to [0,1]
P:S×Y×S×X→[0,1]
\displaystyle\sum_{(s^{\prime} ,y)\in S\times Y} P(s^{\prime}, y|s, x)=1, forall\ (s,x)\in S\times X
​(s​′​​,y)∈S×Y​∑​​P(s​′​​,y∣s,x)=1,forall (s,x)∈S×X
A(y|x)\ with\ a_{ss^{\prime}}(y|x) = P(s',y|s,x)
A(y∣x) with a​ss​′​​​​(y∣x)=P(s​′​​,y∣s,x)

Conditional probabilities can be gathered in collection of stochastic matrices:

Towards a Theory

Special cases of stochastic automata

Markov Chains:

\langle{S, \emptyset, \emptyset, P(s'|s)}\rangle
⟨S,∅,∅,P(s​′​​∣s)⟩
  • Empty input set
  • Empty output set
  • Set of states
  • Transition probabilities from s to s'

Towards a Theory

Special cases of stochastic automata

Stochastic Mealy Automata:

\langle{S, X, Y, P(s', y|s, x)}\rangle
⟨S,X,Y,P(s​′​​,y∣s,x)⟩
P(s', y | s, x) = P(s' | s, x) \cdot P(y | s, x)
P(s​′​​,y∣s,x)=P(s​′​​∣s,x)⋅P(y∣s,x)
P(s'|s,x)= \displaystyle\sum_{y \in Y}P(s', y|s, x)
P(s​′​​∣s,x)=​y∈Y​∑​​P(s​′​​,y∣s,x)
P(y|s,x)= \displaystyle\sum_{s^{\prime} \in S} P(s', y|s, x)
P(y∣s,x)=​s​′​​∈S​∑​​P(s​′​​,y∣s,x)

Where,

forall\ (s', y, s, x) \in S \times Y \times S \times X
forall (s​′​​,y,s,x)∈S×Y×S×X
  • Probability for consecutive state s' and output symbol y  for each input x and state s is mutually independent

Towards a Theory

To model IEC as Stochastic Mealy Automata, need:

\langle{S, X, Y, P(s', y|s, x)}\rangle
⟨S,X,Y,P(s​′​​,y∣s,x)⟩

S = Set of all possible populations of size N

  • Then          has cardinality            <

Y = Function of current state - can be ignored

X = User selection

A(x) = Collection of transition matrices

  • State space
  • Input set
  • Output set
  • Transition matriceses
S^N
S​N​​
\left\vert{S^N}\right\vert \ \ \ \ \infty
​∣​∣​​S​N​​​∣​∣​​    ∞
a_{ss'}(y|x)=P(s',y|s,x)
a​ss​′​​​​(y∣x)=P(s​′​​,y∣s,x)

Towards a Theory

\langle{S, X, Y, P(s', y|s, x)}\rangle
⟨S,X,Y,P(s​′​​,y∣s,x)⟩

A(x) = Transition matrices

  • Let s be current population
  • Let s'' be the user parent population
  • U(x) - matrices of state transitions caused by user
  •  
  • s'' may be changed by crossover/mutation, giving s'
  • These probabilistic changes can be modeled by transition matrix M
  • So, 
A(x)=U(x)\cdot M
A(x)=U(x)⋅M
u_{ss^{\prime \prime}}(x) \in \{0, 1\}
u​ss​′′​​​​(x)∈{0,1}

S = {0, 1}, N = 2

 

U(2,0) 00 01 10 11
00 1 0 0 0
01 1 0 0 0
10 0 0 0 1
11 0 0 0 1

 

 

U(1,1) 00 01 10 11
00 1 0 0 0
01 0 1 0 0
10 0 0 1 0
11 0 0 0 1

 

 

U(0,2) 00 01 10 11
00 0 0 0 1
01 0 0 0 1
10 1 0 0 0
11 1 0 0 0

 

Towards a Theory

\langle{S, X, Y, P(s', y|s, x)}\rangle
⟨S,X,Y,P(s​′​​,y∣s,x)⟩

 

M 00 01 10 11
00 (1-m)^2 m(1-m) m(1-m) m^2
01 1/4 1/4 1/4 1/4
10 1/4 1/4 1/4 1/4
11 m^2 m(1-m) m(1-m) (1-m)^2

 

 

U(2,0) 00 01 10 11
00 1 0 0 0
01 1 0 0 0
10 0 0 0 1
11 0 0 0 1

 

 

A(2,0) 00 01 10 11
00 (1-m)^2 m(1-m) m(1-m) m^2
01 (1-m)^2 m(1-m) m(1-m) m^2
10 m^2 m(1-m) m(1-m) (1-m)^2
11 m^2 m(1-m) m(1-m) (1-m)^2

 

\cdot
⋅
=
=

​(And similarly for the other two matrices)

A(x)=U(x)\cdot M
A(x)=U(x)⋅M

Towards a Theory

Is it useful?

  • Provides formalization that, when given an input sequence, can model the probabilistic behavior of IEC
  • However, input sequences themselves are not deterministic or predictable
  • If they were,             would be function of current state, reducing the model to ordinary Markov chain... regular GA
x \in X
x∈X

Towards a Theory

Is it useful?

It is inconceivable yet to which extent a potential theory of IEAs can profit from stochastic automata theory

Apparently, this work does not exhibit immediate utility with regard to practical applications

Needs more research... this is just the first step

Presentation Overview

 

  1. What is IEC
  2. Motivation
  3. Overview of Field
  4. Technical Aspects
  5. Problems...
  6. Solutions?
  7. Towards a Theory of IGA
  8. Applications and Results of IEC
  9. Conclusion

Image Creation

  • Steven Rooke
  • Maintains library of past good individuals to seed new runs
    • Built up since 1993

 

  • Each pixel from GP function
    • 271 functions used:
      • arithmetic
      • boolean
      • exponential
      • trigonometric
      • fractal
      • randomizers
  • Hundreds of generations to find good images

 

Image Creation

Data Analysis

  • Sergey Malinchik, Eric Bonabeau
  • Applied IEC to data analysis with linear projections
  • Example synthetic data is five dimensional with 24,000 records and 5 clusters
  • Wanted to discover (with no a priori knowledge):
    • Number of clusters
    • Location of clusters
    • Shapes and size of clusters

Data Analysis

  • Linear projections: Wanted to map multidimensional space to 2 dimensional space
  •  
  •  
  •  
  •  
  •  
(x_1, x_2, ..., x_n)\to(x',y')
(x​1​​,x​2​​,...,x​n​​)→(x​′​​,y​′​​)
y' = \sum \beta_ix_i
y​′​​=∑β​i​​x​i​​
x' = \sum \alpha_ix_i
x​′​​=∑α​i​​x​i​​
\sum \alpha_i^2 = 1
∑α​i​2​​=1
\sum \beta_i^2 = 1
∑β​i​2​​=1

Well defined problem, but fitness is still hard to define without further information

Data Analysis

  • Population size = 9
  • Genotype = 
  • Each generation:
    • One elite solution added to next generation
    • Single point crossover used to create 4 new individuals
    • Single point mutation used to create last 4 individuals
  • Results: 10-15 generations for structure of data to become visible
(\alpha_1,...,\alpha_n,\beta_1,...,\beta_n)
(α​1​​,...,α​n​​,β​1​​,...,β​n​​)

Data Analysis

Generation 0

Malinchik, S., Bonabeau, E. Exploratory Data Anlysis with Interactive Evolution

Data Analysis

Generation 15

Malinchik, S., Bonabeau, E. Exploratory Data Anlysis with Interactive Evolution

Music: GenJam

  • IGA for jazz solos
  • Solos played as part of duet with human operator
  • Cooperative 2-level population
    • Measure population and phrase population
    • Measure population maps to MIDI events
    • Phrase population maps to indices in measure population
  • Entire population used to build solo, not just one best individual

Music: GenJam

  • Measure population = 64
  • Phrase population = 48
  • Evaluation:
    1. Half the phrase population is played: 24 four-bar phrases
    2. As user listens, they type 'g' for good, 'b' for bad as many times as they like
    3. Underlying measure and phrase individuals are rewarded as appropriate up to a max and min
  • Modified tournament selection:
    • 4 individuals chosen, two best recombine, to worst replaced by children
  • Check it out!

Presentation Overview

 

  1. What is IEC
  2. Motivation
  3. Overview of Field
  4. Technical Aspects
  5. Problems...
  6. Solutions?
  7. Towards a Theory of IGA
  8. Applications and Results of IEC
  9. Conclusion

Conclusion

  • IEC uses human input for fitness/selection
    • IGA, IGP, IES, HBGA, ISO, etc
  • Problems
    • Constraints: small pop, few generations, few runs
    • Human fatigue
  • ​Solutions
    • Better GUIs
    • nD - 2D mapping of individuals
    • Predictive fitness
    • On-line knowledge embedding
    • Extensions to GA algorithm
  • Can be modeled at Stochastic Mealy Automata
    • Benefit of doing so is not certain, needs more research
  • Many application in domains both creative and otherwise

References

  • Takagi, H. 2001. Interactive evolutionary computation: fusion of the capabilities of EC optimization and human evaluation. Proc. IEEE 89: 1275-1296. 
  • Malinchik, S., Bonabeau, E. 2004. Exploratory Data Analysis with Interactive Evolution. LNCS 3103: 1151-1161.
  • Saez, Y., Isasi, P., Segovia, J. Interactive Evolutionary Computation algorithms applied to solve Rastrigin test functions. 
  • Gunter, R. On Interactive Evolutionary Algorithms and Stochastic Mealy Automata. 
  • Takagi, H., Kishi, K. On-line Knowledge Embedding for an Interactive EC-based Montage System.
  • Takagi, H. Interactive Evolutionary Computation: System Optimization Based on Human Subjective Evaluation.
  • Takagi, H., Hayashida, N. Visualized IEC: Interactive Evolutionary Computation with Multidimensional Data Visualization.
  • Aoki, K., Takagi, H. 3-D CG Lighting with an Interactive GA.
  • Biles, J. A. GenJam: a genetic algorithm for generating jazz solos. Int'l Computer Music Conf. 1994.
  • Steven Rooke. http://srooke.com/

Interactive Evolutionary Computation

By igorii

Made with Slides.com

Interactive Evolutionary Computation

An overview of IEC, its theory, problems, and applications.

  • 1,368

More from igorii