Interactive Evolutionary Design  with
Region-of-Interest Selection  for
Spatiotemporal Ideation & Generation



J Eisenmann


Committee: Rick Parent / Matt Lewis /  Han-Wei Shen



introduction

typical interaction cycle

 

  • display
  • select
  • reproduce
  • repeat until satisfied

Why interactive evolution?

  • simple interface
  • creative support
  • subjective design problems

baseline interactive evolutionary design limitations

    1. Designing motion is more difficult:
      • Long evaluation times for animation
      • Difficulty remembering animations
      • Visual overload
    2. Design spaces with insufficient diversity
    3. Possible detachment or lack of creative ownership
    4. Human evaluation bottleneck



What if designers were given 

the option to choose only 

the parts of phenotypes 

that they like?

sensitive evolutionary design

interactive evolutionary animation

research question 1


Can sensitivity analysis be adapted to identify the sensitive parameters of a digital asset using spatiotemporal selections?


research question 2


Can the resulting sensitivity measures influence the evolutionary algorithm so that region-of-interest selections lead to greater numbers of offspring that have the selected traits than with traditional wholesale selection?

research question 3


Can a visualization interface be developed:

        • for viewing and making decisions about populations of spatiotemporal digital assets?
        • that accommodates human limits on:
          • short-term memory?
          • visual perception?

research question 4


Can an interactive evolutionary selection interface allow users to select regions-of-interest:

        • in both space and time simultaneously?

        • in a way that is both fast and tolerant of imprecise gestures?

baseline pipeline

 

new pipeline

research question 1


Can sensitivity analysis be adapted to identify the sensitive parameters of a digital asset using spatiotemporal selections?





sensitivity analysis

  1. sample the g-dimensional input space
  2. evaluate the model at each sampled point
  3. measure the sensitivity of each parameter

Morris method




Morris Method


Morris Method

Sobol method

practical considerations

  • Morris requires (300x) fewer model evaluations
  • Sobol can measure quantitative sensitivities more accurately 
  • Important variables
    • model evaluation time
    • number of region-of-interest selections
    • number of model evaluations
    • number of parameters

vase parametric model



baseline crossover

baseline mutation


sampling modifications



measuring similarity


  • multiple trajectories
  • relative to root trajectory
  • shape metric
  • translation, rotation, scale invariant
  • temporal matching provided by DTW
  • modifications to suit digital animation



"A new descriptor for multiple 3D motion trajectories recognition by Shao and Li

sum of sincs test model


mutation sensitivity analysis

crossover sensitivity analysis



research question 2


Can the resulting sensitivity measures influence the evolutionary algorithm so that region-of-interest selections lead to greater numbers of offspring that have the selected traits than with traditional wholesale selection?

sensitive crossover


sensitive crossover demo

sensitive mutation


sensitive mutation demo

sensitive reproduction demo

sensitive reproduction results





Population Visualization Challenges



  • visual clutter
  • making comparisons is difficult
  • artificially inflated diversity
  • distracting peripheral motions

research question 3


Can a visualization interface be developed:

        • for viewing and making decisions about populations of spatiotemporal digital assets?
        • that accommodates human limits on:
          • short-term memory?
          • visual perception?

population visualization task description

  • acquire a general sense of the current population
    • the ranges of visible traits
    • the basic categories of designs
  • observe individual designs in detail
  • compare similar designs
  • choose the more interesting designs or characteristics
  • (understand impact of decisions over time)

visual summarization demo


baseline system

IDEAS system

ghost display

streamline display

population layout demo


baseline system

IDEAS system

small multiples layout

RMDS sorted layout

isolation mode demo


baseline system

IDEAS system

playback all

playback selected

memory assistance




region-of-interest selection types


  •  whole

  •  only

  •  especially 

  •  all but

research question 4


Can an interactive evolutionary selection interface allow users to select regions-of-interest:

        • in both space and time simultaneously?

        • in a way that is both fast and tolerant of imprecise gestures?

selecting animated

regions-of-interest

  • not just spatial regions
  • but also time spans
  • typically, this requires context switching

trace selection task description

  • selection interaction should be fast
  • gesture recognition should be tolerant of error
  • real-time feedback on selection
  • minimize context switching

subsequence DTW


trace selection demo

trace selection demo

trace selection user study





case studies

  • holistic, anecdotal evaluation
  • human-in-the-loop
  • designer satisfaction
  • experience notes
  • real world design contexts

case study procedure

  • designers provided the original motion clip
  • a parametric model created with designer input
  • training session
  • design session
  • survey

case study 1

  • digital tailor and simulation artist (Pixar)
  • design praxis: rough to fine
  • streamlines helped her visualize "character" of the animation
  • export and restart strategy
  • surprised by a mutation
  • noticed that outliers are on the edges of the population

case study 2

  • birth sequence for rock giant character in his thesis
  • specific design context
  • two design sessions
    • 1: low diversity space
    • 2: expanded design space
  • surprised by novel animation ideas
  • trace selection interface suggestion

case study 3

  • motion clips for 2 fairy characters in a side scroller game
  • selected 2 foot motions and wanted to see the average
  • grouping in the population layout helped with developing two different motion ideas simultaneously
  • also used the export and restart strategy
  • suggested viewing angle presets

case study summary

  • design session times: 30 minutes to an hour
  • number of generations: 5 to 11
  • all participants 
    • enjoyed the experience of navigating the design space and felt that they could navigate towards interesting designs
    • were satisfied with the results of their process
    • felt that they had arrived at a place in the design process where tweaking could begin
    • changed their minds about their interests mid-design session
    • found at least one unexpected, interesting idea

case study analysis

  • all participants
    • reported that it was easy to notice patterns and prevailing tendencies within the population
    • said making comparisons between similar individuals was easy
    • used the parents display and the selection history at least once
  • animation focus aid answers varied widely
  • trace selection feedback also varied, and differed from user study results (different test bed)

case study analysis (continued)

  • all perceived the region-of-interest selection methods to be either as useful or slightly less useful than whole-candidate selection ("all but" was the least popular)
  • export and restart strategy was common
  • parametric granularity
  • wide variety of application ideas

research questions


    1. Can sensitivity analysis be adapted to identify the sensitive parameters of a digital asset using spatiotemporal selections?

    2. Can the resulting sensitivity measures influence the evolutionary algorithm so that region-of-interest selections lead to greater numbers of offspring that have the selected traits than with traditional wholesale selection?

major contributions


  • Sensitivity analysis has been successfully adapted to identify the sensitive parameters that correspond to spatiotemporal aspects of digital assets. (RQ1)

  • The sensitivity measures have been shown, in side-by-side comparison with the pre-existing system, to influence the evolutionary algorithm so that region-of-interest selections lead to greater numbers of offspring that have the selected traits than with traditional wholesale selection. (RQ2)

research questions (continued)


    1. Can a visualization interface be developed for viewing and making decisions about populations of spatiotemporal digital assets that accommodates human limits on short-term memory and visual perception?

    2. Can an interactive evolutionary selection interface allow users to select regions-of-interest in both space and time simultaneously in a way that is both fast and tolerant of imprecise gestures?

supporting contributions


  • The selection interface user study has shown that trace selection allows users to select regions-of-interest in both space and time simultaneously, and is both fast and enjoyable to use. (RQ4)

  • The population visualization interface for spatiotemporal designs enabled case study participants to view and make decisions about the designs without succumbing to the human limitations of short-term memory and perception bandwidth. (RQ3)

reflective critique


  • faster computation between generations
  • further case studies
  • comparative interface user study

future research


  • similarity measures for other design domains
  • mass customization
  • direct manipulation tweaks during evolution
  • parametric modeling toolkit and best practices
  • sensitive non-interactive evolutionary computation 

conclusion

  • sensitivity analysis works
    • for hierarchical, animated digital assets
    • with interactive evolutionary design
  • novel interface allows for
    • viewing sets of animations
    • selection of spatiotemporal regions
  • whole system tested in real world design contexts



Cara Malek



J Eisenmann



J Eisenmann


Matt Lewis


Matt Lewis


Matt Lewis


J Eisenmann

example design session


>>

example design session


example design session


example design session


example design session


example design session


example design session


example design session


example design session


example design session


example design session



interactive evolutionary design

applications


  • ideation
  • generation
  • novice design
  • memory reconstruction






>

roadblocks to production use

  • lack of familiarity
  • creative control
  • human evaluation bottleneck

types of sensitivity analysis


  1. screening methods*
  2. differential analysis methods
  3. Monte Carlo methods*

balancing exploration & exploitation

  • population diversity depends heavily on selections
  • satisfactory vs premature convergence
  • additional controls:

design problem scope

human evaluation challenges

  • fitness feedback
  • visual overload
  • memory overload
  • total evaluation time
  • wait time between generations

Defense

By jeisenma

Defense

  • 551
Loading comments...

More from jeisenma