Interactive Evolutionary Design with

Region-of-Interest Selection for

Spatiotemporal Ideation & Generation

J Eisenmann

Committee: Rick Parent / Matt Lewis / Han-Wei Shen

introduction

interactive evolutionary design

applications

ideation
generation
novice design
memory reconstruction

typical interaction cycle

display
select
reproduce
repeat until satisfied

example design session

Why interactive evolution?

simple interface
creative support
subjective design problems

roadblocks to production use

lack of familiarity
creative control
human evaluation bottleneck

human evaluation challenges

total evaluation time
fitness feedback
visual overload
memory overload
wait time between generations

baseline crossover

baseline mutation

baseline interactive evolutionary design limitations

Designing motion is more difficult:
- Long evaluation times for animation
- Difficulty remembering animations
- Visual overload
Design spaces with insufficient diversity
Possible detachment or lack of creative ownership
Parametric modeling

What if designers were given

the option to choose only

the parts of phenotypes

that they like?

sensitive evolutionary design

design problem scope

interactive evolutionary animation

research question 1

Can sensitivity analysis be adapted to identify the sensitive parameters of a digital asset using spatiotemporal selections?

research question 2

Can the resulting sensitivity measures influence the evolutionary algorithm so that region-of-interest selections lead to greater numbers of offspring that have the selected traits than with traditional wholesale selection?

research question 3

Can a visualization interface be developed:

for viewing and making decisions about
populations of spatiotemporal digital assets
that accommodates human limits on:

short-term memory
visual perception?

research question 4

Can an interactive evolutionary selection interface:

allow users to select regions-of-interest
in both space and time simultaneously
in a way that is both fast and tolerant of imprecise gestures?

pipeline

research question 1

Can sensitivity analysis be adapted to identify the sensitive parameters of a digital asset using spatiotemporal selections?

sensitivity analysis

sample the g-dimensional input space
evaluate the model at each sampled point
measure the sensitivity of each parameter

types of sensitivity analysis

screening methods*
differential analysis methods
Monte Carlo methods*

Morris method

Morris Method

Morris Method

Sobol method

practical considerations

Morris requires (300x) fewer model evaluations
Sobol can measure quantitative sensitivities more accurately
Important variables

model evaluation time
number of region-of-interest selections
number of model evaluations
number of genes

vase parametric model

sampling modifications

measuring similarity

multiple trajectories
relative to root trajectory
shape metric
translation, rotation, scale invariant
temporal matching provided by DTW
modifications to suit digital animation

"A new descriptor for multiple 3D motion trajectories recognition" by Shao and Li

sum of sincs test model

mutation sensitivity analysis

crossover sensitivity analysis

research question 2

sensitive crossover

sensitive crossover demo

sensitive mutation

sensitive mutation demo

sensitive reproduction demo

sensitive reproduction results

balancing exploration & exploitation

population diversity depends heavily on selections
satisfactory vs premature convergence
additional controls:

Population Visualization Challenges

visual clutter
making comparisons is difficult
artificially inflated diversity
distracting peripheral motions

research question 3

Can a visualization interface be developed:

for viewing and making decisions about
populations of spatiotemporal digital assets
that accommodates human limits on:

short-term memory
visual perception?

population visualization task description

observe individual designs in detail
acquire a general sense of the current population

the ranges of visible traits
the basic categories of designs

compare similar designs
choose the more interesting designs or characteristics
(understand impact of decisions over time)

visual summarization demo

population layout demo

isolation mode demo

memory assistance

region-of-interest selection types

whole
only
especially
all but

research question 4

Can an interactive evolutionary selection interface:

allow users to select regions-of-interest
in both space and time simultaneously
in a way that is both fast and tolerant of imprecise gestures?

selecting animated

regions-of-interest

not just spatial regions
but also time spans
typically, this requires context switching

trace selection task description

selection interaction should be fast
gesture recognition should be tolerant of error
precise spatial selection is not necessary
real-time feedback on selection
minimize context switching

subsequence DTW

trace selection demo

trace selection user study

case studies

holistic evaluation
human-in-the-loop
designer satisfaction
experience notes
design context

case study procedure

designers provided the original motion clip
a parametric model was created with designer input
training session
design session
survey

case study 1

digital tailor and simulation artist (Pixar)
design praxis: rough to fine
streamlines helped her visualize "character" of the animation
export and restart strategy
surprised by a mutation
noticed that outliers are on the edges of the population

case study 2

birth sequence for rock giant character in his thesis
very specific design context
limited parametric possibilities
two design sessions

1: low diversity space
2: expanded design space

surprised by novel animation ideas
whole trace selection interface suggestion

case study 3

motion clips for 2 fairy characters in a side scroller game
selected 2 foot motions and wanted to see the average
grouping in the population layout helped with developing two different motion ideas simultaneously
also used the export and restart strategy
suggested viewing angle presets

case study summary

design session times: 30 minutes to an hour
number of generations: 5 to 11
all participants

enjoyed the experience of navigating the design space and felt that they could navigate towards interesting designs
were satisfied with the results of their process
felt that they had arrived at a place in the design process where tweaking could begin
changed their minds about their interests mid-design session
found at least one unexpected, interesting idea

case study analysis

all participants

reported that it was easy to notice patterns and prevailing tendencies within the population
said making comparisons between similar individuals was easy
used the parents display and the selection history at least once

animation focus aid answers varied widely
trace selection feedback also varied, and differed from user study results (different test bed)

case study analysis (continued)

all perceived the region-of-interest selection methods to be either as useful or slightly less useful than whole-candidate selection ("all but" was the least popular)
export and restart strategy was common
parametric granularity
wide variety of application ideas

research questions

Can sensitivity analysis be adapted to identify the sensitive parameters of a digital asset using spatiotemporal selections?
Can the resulting sensitivity measures influence the evolutionary algorithm so that region-of-interest selections lead to greater numbers of offspring that have the selected traits than with traditional wholesale selection?

research questions (continued)

Can a visualization interface be developed for viewing and making decisions about populations of spatiotemporal digital assets that accommodates human limits on short-term memory and visual perception?
Can an interactive evolutionary selection interface allow users to select regions-of-interest in both space and time simultaneously in a way that is both fast and tolerant of imprecise gestures?

major contributions

sensitivity analysis has been successfully adapted to identify the sensitive parameters that correspond to spatiotemporal aspects of digital assets (RQ1)
the sensitivity measures have been shown, in side-by-side comparison with the pre-existing system, to influence the evolutionary algorithm so that region-of-interest selections lead to greater numbers of offspring that have the selected traits than with traditional wholesale selection (RQ2)

supporting contributions

selection interface user study has shown that trace selection allows users to select regions-of-interest in both space and time simultaneously, and is both fast and tolerant of imprecise gestures (RQ4)
population visualization interface for spatiotemporal designs enables users to view and make decisions about the designs without succumbing to the human limitations of short-term memory and perception bandwidth (RQ3)