Interactive Evolutionary Design with
Region-of-Interest Selection for
Spatiotemporal Ideation & Generation



J Eisenmann


Committee: Rick Parent / Matt Lewis / Han-Wei Shen



introduction

interactive evolutionary design

applications


  • ideation
  • generation
  • novice design
  • memory reconstruction






>

typical interaction cycle

 

  • display
  • select
  • reproduce
  • repeat until satisfied

example design session


>>

Why interactive evolution?

  • simple interface
  • creative support
  • subjective design problems

roadblocks to production use

  • lack of familiarity
  • creative control
  • human evaluation bottleneck

human evaluation challenges

  • total evaluation time
  • fitness feedback
  • visual overload
  • memory overload
  • wait time between generations

baseline crossover

baseline mutation


baseline interactive evolutionary design limitations

    1. Designing motion is more difficult:
      • Long evaluation times for animation
      • Difficulty remembering animations
      • Visual overload
    2. Design spaces with insufficient diversity
    3. Possible detachment or lack of creative ownership
    4. Parametric modeling



What if designers were given 

the option to choose only 

the parts of phenotypes 

that they like?

sensitive evolutionary design

design problem scope

interactive evolutionary animation

research question 1


Can sensitivity analysis be adapted to identify the sensitive parameters of a digital asset using spatiotemporal selections?


research question 2


Can the resulting sensitivity measures influence the evolutionary algorithm so that region-of-interest selections lead to greater numbers of offspring that have the selected traits than with traditional wholesale selection?

research question 3


Can a visualization interface be developed:

        • for viewing and making decisions about
        • populations of spatiotemporal digital assets
        • that accommodates human limits on:
          • short-term memory
          • visual perception?

research question 4


Can an interactive evolutionary selection interface:

        • allow users to select regions-of-interest

        • in both space and time simultaneously

        • in a way that is both fast and tolerant of imprecise gestures?

pipeline

research question 1


Can sensitivity analysis be adapted to identify the sensitive parameters of a digital asset using spatiotemporal selections?





sensitivity analysis

  1. sample the g-dimensional input space
  2. evaluate the model at each sampled point
  3. measure the sensitivity of each parameter

types of sensitivity analysis


  1. screening methods*
  2. differential analysis methods
  3. Monte Carlo methods*

Morris method




Morris Method


Morris Method

Sobol method

practical considerations

  • Morris requires (300x) fewer model evaluations
  • Sobol can measure quantitative sensitivities more accurately 
  • Important variables
    • model evaluation time
    • number of region-of-interest selections
    • number of model evaluations
    • number of genes

vase parametric model



sampling modifications



measuring similarity


  • multiple trajectories
  • relative to root trajectory
  • shape metric
  • translation, rotation, scale invariant
  • temporal matching provided by DTW
  • modifications to suit digital animation



"A new descriptor for multiple 3D motion trajectories recognition"  by Shao and Li

sum of sincs test model


mutation sensitivity analysis

crossover sensitivity analysis



research question 2


Can the resulting sensitivity measures influence the evolutionary algorithm so that region-of-interest selections lead to greater numbers of offspring that have the selected traits than with traditional wholesale selection?

sensitive crossover


sensitive crossover demo

sensitive mutation


sensitive mutation demo

sensitive reproduction demo

sensitive reproduction results


balancing exploration & exploitation

  • population diversity depends heavily on selections
  • satisfactory vs premature convergence
  • additional controls:




Population Visualization Challenges



  • visual clutter
  • making comparisons is difficult
  • artificially inflated diversity
  • distracting peripheral motions

research question 3


Can a visualization interface be developed:

        • for viewing and making decisions about
        • populations of spatiotemporal digital assets
        • that accommodates human limits on:
          • short-term memory
          • visual perception?

population visualization task description

  • observe individual designs in detail
  • acquire a general sense of the current population
    • the ranges of visible traits
    • the basic categories of designs
  • compare similar designs
  • choose the more interesting designs or characteristics
  • (understand impact of decisions over time)

visual summarization demo


population layout demo


isolation mode demo


memory assistance




region-of-interest selection types


  •  whole

  •  only

  •  especially 

  •  all but

research question 4


Can an interactive evolutionary selection interface:

        • allow users to select regions-of-interest

        • in both space and time simultaneously

        • in a way that is both fast and tolerant of imprecise gestures?

selecting animated

regions-of-interest

  • not just spatial regions
  • but also time spans
  • typically, this requires context switching

trace selection task description

  • selection interaction should be fast
  • gesture recognition should be tolerant of error
  • precise spatial selection is not necessary
  • real-time feedback on selection
  • minimize context switching

subsequence DTW


trace selection demo

trace selection demo

trace selection user study


case studies

  • holistic evaluation
  • human-in-the-loop
  • designer satisfaction
  • experience notes
  • design context

case study procedure

  • designers provided the original motion clip
  • a parametric model was created with designer input
  • training session
  • design session
  • survey

case study 1

  • digital tailor and simulation artist (Pixar)
  • design praxis: rough to fine
  • streamlines helped her visualize "character" of the animation
  • export and restart strategy
  • surprised by a mutation
  • noticed that outliers are on the edges of the population

case study 2

  • birth sequence for rock giant character in his thesis
  • very specific design context
  • limited parametric possibilities
  • two design sessions
    • 1: low diversity space
    • 2: expanded design space
  • surprised by novel animation ideas
  • whole trace selection interface suggestion

case study 3

  • motion clips for 2 fairy characters in a side scroller game
  • selected 2 foot motions and wanted to see the average
  • grouping in the population layout helped with developing two different motion ideas simultaneously
  • also used the export and restart strategy
  • suggested viewing angle presets

case study summary

  • design session times: 30 minutes to an hour
  • number of generations: 5 to 11
  • all participants 
    • enjoyed the experience of navigating the design space and felt that they could navigate towards interesting designs
    • were satisfied with the results of their process
    • felt that they had arrived at a place in the design process where tweaking could begin
    • changed their minds about their interests mid-design session
    • found at least one unexpected, interesting idea

case study analysis

  • all participants
    • reported that it was easy to notice patterns and prevailing tendencies within the population
    • said making comparisons between similar individuals was easy
    • used the parents display and the selection history at least once
  • animation focus aid answers varied widely
  • trace selection feedback also varied, and differed from user study results (different test bed)

case study analysis (continued)

  • all perceived the region-of-interest selection methods to be either as useful or slightly less useful than whole-candidate selection ("all but" was the least popular)
  • export and restart strategy was common
  • parametric granularity
  • wide variety of application ideas

research questions


    1. Can sensitivity analysis be adapted to identify the sensitive parameters of a digital asset using spatiotemporal selections?

    2. Can the resulting sensitivity measures influence the evolutionary algorithm so that region-of-interest selections lead to greater numbers of offspring that have the selected traits than with traditional wholesale selection?

research questions (continued)


    1. Can a visualization interface be developed for viewing and making decisions about populations of spatiotemporal digital assets that accommodates human limits on short-term memory and visual perception?

    2. Can an interactive evolutionary selection interface allow users to select regions-of-interest in both space and time simultaneously in a way that is both fast and tolerant of imprecise gestures?

major contributions


  • sensitivity analysis has been successfully adapted to identify the sensitive parameters that correspond to spatiotemporal aspects of digital assets (RQ1)

  • the sensitivity measures have been shown, in side-by-side comparison with the pre-existing system, to influence the evolutionary algorithm so that region-of-interest selections lead to greater numbers of offspring that have the selected traits than with traditional wholesale selection (RQ2)

supporting contributions


  • selection interface user study has shown that trace selection allows users to select regions-of-interest in both space and time simultaneously, and is both fast and tolerant of imprecise gestures (RQ4)

  • population visualization interface for spatiotemporal designs enables users to view and make decisions about the designs without succumbing to the human limitations of short-term memory and perception bandwidth (RQ3)

reflective critique


  • faster computation
  • further case studies
  • comparative interface study

future research


  • similarity measures for other design domains
  • mass customization
  • direct manipulation tweaks during evolution
  • parametric modeling toolkit and best practices
  • sensitive non-interactive evolutionary computation 

conclusion

  • sensitivity analysis
    • for hierarchical, animated digital assets
    • with interactive evolutionary design
  • novel interface
    • viewing sets of animations
    • selection spatiotemporal regions
  • tested in real-world design contexts



Cara Malek



J Eisenmann



J Eisenmann


Matt Lewis


Matt Lewis


Matt Lewis


J Eisenmann

example design session


example design session


example design session


example design session


example design session


example design session


example design session


example design session


example design session


example design session


example design session


Made with Slides.com