Interactive Evolutionary Design
with
Region-of-Interest Selection
for
Spatiotemporal Ideation & Generation
J Eisenmann
Committee: Rick Parent / Matt Lewis / Han-Wei Shen
introduction
typical interaction cycle
- display
- select
- reproduce
- repeat until satisfied
Why interactive evolution?
-
simple interface
-
creative support
-
subjective design problems
baseline interactive evolutionary design limitations
- Designing motion is more difficult:
- Long evaluation times for animation
- Difficulty remembering animations
- Visual overload
- Design spaces with insufficient diversity
- Possible detachment or lack of creative ownership
- Human evaluation bottleneck
What if designers were given
the option to choose only
the parts of phenotypes
that they like?
sensitive evolutionary design
interactive evolutionary animation
research question 1
Can sensitivity analysis be adapted to identify the sensitive parameters of a digital asset using spatiotemporal selections?
research question 2
Can the resulting sensitivity measures influence the evolutionary algorithm so that region-of-interest selections lead to greater numbers of offspring that have the selected traits than with traditional wholesale selection?
research question 3
Can a visualization interface be developed:
- for viewing and making decisions about populations of spatiotemporal digital assets?
- that accommodates human limits on:
- short-term memory?
- visual perception?
research question 4
Can an interactive evolutionary selection interface allow users to select regions-of-interest:
in both space and time simultaneously?
-
in a way that is both fast and tolerant of imprecise gestures?
baseline pipeline
new pipeline
research question 1
Can sensitivity analysis be adapted to identify the sensitive parameters of a digital asset using spatiotemporal selections?
sensitivity analysis
- sample the g-dimensional input space
- evaluate the model at each sampled point
- measure the sensitivity of each parameter
Morris method
Morris Method
Morris Method
Sobol method
practical considerations
- Morris requires (300x) fewer model evaluations
- Sobol can measure quantitative sensitivities more accurately
- Important variables
- model evaluation time
- number of region-of-interest selections
- number of model evaluations
- number of parameters
vase parametric model
baseline crossover
baseline mutation
sampling modifications
measuring similarity
- multiple trajectories
- relative to root trajectory
- shape metric
- translation, rotation, scale invariant
- temporal matching provided by DTW
- modifications to suit digital animation
"A new descriptor for multiple 3D motion trajectories recognition"
by Shao and Li
sum of sincs test model
mutation sensitivity analysis
crossover sensitivity analysis
research question 2
Can the resulting sensitivity measures influence the evolutionary algorithm so that region-of-interest selections lead to greater numbers of offspring that have the selected traits than with traditional wholesale selection?
sensitive crossover
sensitive crossover demo
sensitive mutation
sensitive mutation demo
sensitive reproduction demo
sensitive reproduction results
Population Visualization Challenges
-
visual clutter
-
making comparisons is difficult
-
artificially inflated diversity
-
distracting peripheral motions
research question 3
Can a visualization interface be developed:
- for viewing and making decisions about populations of spatiotemporal digital assets?
- that accommodates human limits on:
- short-term memory?
- visual perception?
population visualization task description
- acquire a general sense of the current population
- the ranges of visible traits
- the basic categories of designs
- observe individual designs in detail
- compare similar designs
- choose the more interesting designs or characteristics
- (understand impact of decisions over time)
visual summarization demo
baseline system
IDEAS system
ghost display
streamline display
population layout demo
baseline system
IDEAS system
small multiples layout
RMDS sorted layout
isolation mode demo
baseline system
IDEAS system
playback all
playback selected
memory assistance
region-of-interest selection types
-
whole
-
only
-
especially
-
all but
research question 4
Can an interactive evolutionary selection interface allow users to select regions-of-interest:
-
in both space and time simultaneously?
-
in a way that is both fast and tolerant of imprecise gestures?
selecting animated
regions-of-interest
- not just spatial regions
- but also time spans
- typically, this requires context switching
trace selection task description
- selection interaction should be fast
- gesture recognition should be tolerant of error
-
real-time feedback on selection
- minimize context switching
subsequence DTW
trace selection demo
trace selection demo
trace selection user study
case studies
- holistic, anecdotal evaluation
- human-in-the-loop
- designer satisfaction
- experience notes
- real world design contexts
case study procedure
- designers provided the original motion clip
- a parametric model created with designer input
- training session
- design session
- survey
case study 1
- digital tailor and simulation artist (Pixar)
- design praxis: rough to fine
- streamlines helped her visualize "character" of the animation
- export and restart strategy
- surprised by a mutation
- noticed that outliers are on the edges of the population
case study 2
- birth sequence for rock giant character in his thesis
- specific design context
- two design sessions
- 1: low diversity space
- 2: expanded design space
- surprised by novel animation ideas
- trace selection interface suggestion
case study 3
- motion clips for 2 fairy characters in a side scroller game
- selected 2 foot motions and wanted to see the average
- grouping in the population layout helped with developing two different motion ideas simultaneously
- also used the export and restart strategy
- suggested viewing angle presets
case study summary
- design session times: 30 minutes to an hour
- number of generations: 5 to 11
- all participants
- enjoyed the experience of navigating the design space and felt that they could navigate towards interesting designs
- were satisfied with the results of their process
- felt that they had arrived at a place in the design process where tweaking could begin
- changed their minds about their interests mid-design session
- found at least one unexpected, interesting idea
case study analysis
- all participants
- reported that it was easy to notice patterns and prevailing tendencies within the population
- said making comparisons between similar individuals was easy
- used the parents display and the selection history at least once
- animation focus aid answers varied widely
- trace selection feedback also varied, and differed from user study results (different test bed)
case study analysis (continued)
- all perceived the region-of-interest selection methods to be either as useful or slightly less useful than whole-candidate selection ("all but" was the least popular)
- export and restart strategy was common
- parametric granularity
- wide variety of application ideas
research questions
- Can sensitivity analysis be adapted to identify the sensitive parameters of a digital asset using spatiotemporal selections?
- Can the resulting sensitivity measures influence the evolutionary algorithm so that region-of-interest selections lead to greater numbers of offspring that have the selected traits than with traditional wholesale selection?
major contributions
- Sensitivity analysis has been successfully adapted to identify the sensitive parameters that correspond to spatiotemporal aspects of digital assets. (RQ1)
- The sensitivity measures have been shown, in side-by-side comparison with the pre-existing system, to influence the evolutionary algorithm so that region-of-interest selections lead to greater numbers of offspring that have the selected traits than with traditional wholesale selection. (RQ2)
research questions (continued)
- Can a visualization interface be developed for viewing and making decisions about populations of spatiotemporal digital assets that accommodates human limits on short-term memory and visual perception?
- Can an interactive evolutionary selection interface allow users to select regions-of-interest in both space and time simultaneously in a way that is both fast and tolerant of imprecise gestures?
supporting contributions
- The selection interface user study has shown that trace selection allows users to select regions-of-interest in both space and time simultaneously, and is both fast and enjoyable to use. (RQ4)
- The population visualization interface for spatiotemporal designs enabled case study participants to view and make decisions about the designs without succumbing to the human limitations of short-term memory and perception bandwidth. (RQ3)
reflective critique
- faster computation between generations
- further case studies
- comparative interface user study
future research
- similarity measures for other design domains
- mass customization
- direct manipulation tweaks during evolution
- parametric modeling toolkit and best practices
- sensitive non-interactive evolutionary computation
conclusion
- sensitivity analysis works
- for hierarchical, animated digital assets
- with interactive evolutionary design
- novel interface allows for
- viewing sets of animations
- selection of spatiotemporal regions
- whole system tested in real world design contexts
Cara Malek
J Eisenmann
J Eisenmann
Matt Lewis
Matt Lewis
Matt Lewis
J Eisenmann
example design session
>>
example design session
example design session
example design session
example design session
example design session
example design session
example design session
example design session
example design session
example design session
interactive evolutionary design
applications
- ideation
- generation
- novice design
- memory reconstruction
>
roadblocks to production use
-
lack of familiarity
-
creative control
-
human evaluation bottleneck
types of sensitivity analysis
- screening methods*
- differential analysis methods
- Monte Carlo methods*
balancing exploration & exploitation
- population diversity depends heavily on selections
- satisfactory vs premature convergence
- additional controls:
design problem scope
human evaluation challenges
- fitness feedback
- visual overload
- memory overload
- total evaluation time
- wait time between generations
Interactive Evolutionary Design with Region-of-Interest Selection for Spatiotemporal Ideation & Generation J Eise nmann Committee: Rick Parent / Matt Lewis / Han-Wei Shen
Defense
By jeisenma
Defense
- 3,069
