Interactive Evolutionary Design with

Region-of-Interest Selection for

Spatiotemporal Ideation & Generation

J Eisenmann

Committee: Rick Parent / Matt Lewis / Han-Wei Shen

introduction

typical interaction cycle

display
select
reproduce
repeat until satisfied

Why interactive evolution?

simple interface
creative support
subjective design problems

baseline interactive evolutionary design limitations

Designing motion is more difficult:
- Long evaluation times for animation
- Difficulty remembering animations
- Visual overload
Design spaces with insufficient diversity
Possible detachment or lack of creative ownership
Human evaluation bottleneck

What if designers were given

the option to choose only

the parts of phenotypes

that they like?

sensitive evolutionary design

interactive evolutionary animation

research question 1

Can sensitivity analysis be adapted to identify the sensitive parameters of a digital asset using spatiotemporal selections?

research question 2

Can the resulting sensitivity measures influence the evolutionary algorithm so that region-of-interest selections lead to greater numbers of offspring that have the selected traits than with traditional wholesale selection?

research question 3

Can a visualization interface be developed:

for viewing and making decisions about populations of spatiotemporal digital assets?
that accommodates human limits on:

short-term memory?
visual perception?

research question 4

Can an interactive evolutionary selection interface allow users to select regions-of-interest:

in both space and time simultaneously?
in a way that is both fast and tolerant of imprecise gestures?

baseline pipeline

new pipeline

research question 1

Can sensitivity analysis be adapted to identify the sensitive parameters of a digital asset using spatiotemporal selections?

sensitivity analysis

sample the g-dimensional input space
evaluate the model at each sampled point
measure the sensitivity of each parameter

Morris method

Morris Method

Morris Method

Sobol method

practical considerations

Morris requires (300x) fewer model evaluations
Sobol can measure quantitative sensitivities more accurately
Important variables

model evaluation time
number of region-of-interest selections
number of model evaluations
number of parameters

vase parametric model

baseline crossover

baseline mutation

sampling modifications

measuring similarity

multiple trajectories
relative to root trajectory
shape metric
translation, rotation, scale invariant
temporal matching provided by DTW
modifications to suit digital animation

"A new descriptor for multiple 3D motion trajectories recognition" by Shao and Li

sum of sincs test model

mutation sensitivity analysis

crossover sensitivity analysis

research question 2

sensitive crossover

sensitive crossover demo

sensitive mutation

sensitive mutation demo

sensitive reproduction demo

sensitive reproduction results

Population Visualization Challenges

visual clutter
making comparisons is difficult
artificially inflated diversity
distracting peripheral motions

research question 3

Can a visualization interface be developed:

for viewing and making decisions about populations of spatiotemporal digital assets?
that accommodates human limits on:

short-term memory?
visual perception?

population visualization task description

acquire a general sense of the current population

the ranges of visible traits
the basic categories of designs

observe individual designs in detail
compare similar designs
choose the more interesting designs or characteristics
(understand impact of decisions over time)

visual summarization demo

baseline system

IDEAS system

ghost display

streamline display

population layout demo

baseline system

IDEAS system

small multiples layout

RMDS sorted layout

isolation mode demo

baseline system

IDEAS system

playback all

playback selected

memory assistance

region-of-interest selection types

whole
only
especially
all but

research question 4

Can an interactive evolutionary selection interface allow users to select regions-of-interest:

in both space and time simultaneously?
in a way that is both fast and tolerant of imprecise gestures?

selecting animated

regions-of-interest

not just spatial regions
but also time spans
typically, this requires context switching

trace selection task description

selection interaction should be fast
gesture recognition should be tolerant of error
real-time feedback on selection
minimize context switching

subsequence DTW

trace selection demo

trace selection user study

case studies

holistic, anecdotal evaluation
human-in-the-loop
designer satisfaction
experience notes
real world design contexts

case study procedure

designers provided the original motion clip
a parametric model created with designer input
training session
design session
survey

case study 1

digital tailor and simulation artist (Pixar)
design praxis: rough to fine
streamlines helped her visualize "character" of the animation
export and restart strategy
surprised by a mutation
noticed that outliers are on the edges of the population

case study 2

birth sequence for rock giant character in his thesis
specific design context
two design sessions

1: low diversity space
2: expanded design space

surprised by novel animation ideas
trace selection interface suggestion

case study 3

motion clips for 2 fairy characters in a side scroller game
selected 2 foot motions and wanted to see the average
grouping in the population layout helped with developing two different motion ideas simultaneously
also used the export and restart strategy
suggested viewing angle presets

case study summary

design session times: 30 minutes to an hour
number of generations: 5 to 11
all participants

enjoyed the experience of navigating the design space and felt that they could navigate towards interesting designs
were satisfied with the results of their process
felt that they had arrived at a place in the design process where tweaking could begin
changed their minds about their interests mid-design session
found at least one unexpected, interesting idea

case study analysis

all participants

reported that it was easy to notice patterns and prevailing tendencies within the population
said making comparisons between similar individuals was easy
used the parents display and the selection history at least once

animation focus aid answers varied widely
trace selection feedback also varied, and differed from user study results (different test bed)

case study analysis (continued)

all perceived the region-of-interest selection methods to be either as useful or slightly less useful than whole-candidate selection ("all but" was the least popular)
export and restart strategy was common
parametric granularity
wide variety of application ideas

research questions

Can sensitivity analysis be adapted to identify the sensitive parameters of a digital asset using spatiotemporal selections?
Can the resulting sensitivity measures influence the evolutionary algorithm so that region-of-interest selections lead to greater numbers of offspring that have the selected traits than with traditional wholesale selection?

major contributions

Sensitivity analysis has been successfully adapted to identify the sensitive parameters that correspond to spatiotemporal aspects of digital assets. (RQ1)
The sensitivity measures have been shown, in side-by-side comparison with the pre-existing system, to influence the evolutionary algorithm so that region-of-interest selections lead to greater numbers of offspring that have the selected traits than with traditional wholesale selection. (RQ2)

research questions (continued)

Can a visualization interface be developed for viewing and making decisions about populations of spatiotemporal digital assets that accommodates human limits on short-term memory and visual perception?
Can an interactive evolutionary selection interface allow users to select regions-of-interest in both space and time simultaneously in a way that is both fast and tolerant of imprecise gestures?

supporting contributions

The selection interface user study has shown that trace selection allows users to select regions-of-interest in both space and time simultaneously, and is both fast and enjoyable to use. (RQ4)
The population visualization interface for spatiotemporal designs enabled case study participants to view and make decisions about the designs without succumbing to the human limitations of short-term memory and perception bandwidth. (RQ3)