Exploring accelerated evolutionary parameter search for iterative large-scale transport simulations in a new calibration testbed

Sebastian Hörl

3 June 2022

hEART 2022

Challenge

• Agent-based models have become a common tool in transport planning
   

• Reproducibility: Are simulation results replicable by other researchers?
   

• Relevancy: Are the results corresponding well to reality?
   

• Reuse: Can we easily adapt and advance the modeling system?

Modeling pipeline

• Goal: Provide a modeling pipeline from raw data to the final results.

• Should allow to automatically repeat the modeling process.
• Should allow to replace individual modeling components.

Synthetic population

Synthetic population

+

=

Synthetic population

+

=

Community

eqasim-org/eqasim-java

eqasim-org/ile-de-france

Modeling pipeline

Modeling pipeline

Simulation

Synthetic demand

Simulation

Mobility simulation

Synthetic demand

Simulation

Decision-making

10:00 - 17:30

17:45 - 21:00

22:00 - 0:00

Mobility simulation

Synthetic demand

Simulation

Decision-making

Mobility simulation

Synthetic demand

Simulation

Decision-making

Mobility simulation

Synthetic demand

Simulation

Decision-making

Mobility simulation

Synthetic demand

Simulation

Decision-making

Mobility simulation

Synthetic demand

Simulation

Decision-making

Mobility simulation

Analysis

Synthetic demand

Calibration problem

• Find parameters (mode choice model, network capacities, ...) that minimize mismatch with reference data

Calibration loop

Calibration loop

Stopping criterion

• Currently, simple process:
  • Smooth trajectory of mode shares over horizon S
     
  • Approximate derivative over horizon H
     
  • Define threshold

Example

• Parameters: ASCs, VOT Car, scaling factor on network capacity
• Target: Mode share by distance
• Reference: EGT 2010 Île-de-France
• Objective: L2 norm
• Algorithm: CMA-(1,1)-ES

Example

• Parameters: ASCs, VOT Car, scaling factor on network capacity
• Target: Mode share by distance
• Reference: EGT 2010 Île-de-France
• Objective: L2 norm
• Algorithm: CMA-(1,1)-ES

Framework

• Designed as a modular framework
  • Wraps around different simulators
  • Modular structure of objectives
  • Various optimization algorithms
  • (Unit tested)
  • (Restartable)

sebhoerl/boptx

Framework

• Gradient approximation

  • SPSA
  • FDSA

• Evolutionary search

  • CMA-ES / Elitist CMA-ES
  • xNES / Elitist xNES

• Accelleration

  • odpyts

• Others

  • Uniform sampling
  • Nelder-Mead
  • Differential Evolution
  • scipy optimize interface

• Coming up

  • Batch Bayesian Optimization (Kriging) with various acquisition functions
  • Multi-fidelity BBO

Framework

• Gradient approximation

  • SPSA
  • FDSA

• Evolutionary search

  • CMA-ES / Elitist CMA-ES
  • xNES / Elitist xNES

• Accelleration

  • odpyts

• Others

  • Uniform sampling
  • Nelder-Mead
  • Differential Evolution
  • scipy optimize interface

• Coming up

  • Batch Bayesian Optimization (Kriging) with various acquisition functions
  • Multi-fidelity BBO

Testing opdyts

• Algorithm developed by Flötteröd (2017)
  
  
   
• Takes into account iterative structure of MATSim-like simulators

Flötteröd, G. (2017) A search acceleration method for optimization problems with
transport simulation constraints, Transportation Research Part B: Methodological, 98, 239-260

Sample N

candidates

Testing opdyts

• Algorithm developed by Flötteröd (2017)
  
  
   
• Takes into account iterative structure of MATSim-like simulators

Flötteröd, G. (2017) A search acceleration method for optimization problems with
transport simulation constraints, Transportation Research Part B: Methodological, 98, 239-260

Sample N

candidates

Run for T

iterations

Testing opdyts

• Algorithm developed by Flötteröd (2017)
  
  
   
• Takes into account iterative structure of MATSim-like simulators

Flötteröd, G. (2017) A search acceleration method for optimization problems with
transport simulation constraints, Transportation Research Part B: Methodological, 98, 239-260

Sample N

candidates

Run for T

iterations

Advance for T

iterations based on transient performance

Testing opdyts

• Algorithm developed by Flötteröd (2017)
  
  
   
• Takes into account iterative structure of MATSim-like simulators

Flötteröd, G. (2017) A search acceleration method for optimization problems with
transport simulation constraints, Transportation Research Part B: Methodological, 98, 239-260

Sample N

candidates

Run for T

iterations

Advance for T

iterations based on transient performance

Until one candidate has converged

Testing opdyts

• Algorithm developed by Flötteröd (2017)
  
  
   
• Takes into account iterative structure of MATSim-like simulators

Flötteröd, G. (2017) A search acceleration method for optimization problems with
transport simulation constraints, Transportation Research Part B: Methodological, 98, 239-260

Sample N

candidates

Run for T

iterations

Advance for T

iterations based on transient performance

Until one candidate has converged

Testing opdyts

• opdyts can now be tested in combination with any algorithm that
  • Allows to generate N candiates
  • Receives one selected candidate

CMA-(1,1)-ES

Opdyts

Candidates

Kriging

Neighborhood

Testing opdyts

• Comparison with CMA-(1,1)-ES
• Speed-up with limited resources
• Otherwise lack of parallelization

Outlook: Testing objectives

• Mode shares (by distance, by time)
  • Household travel surveys
     
• Network flows (daily, hourly)
  • Open Paris Data
     
• Zonal travel times (daily, hourly)
  • Uber Movements data
     
• Transit ridership
  • Navigo validation data

Outlook: Testing combinations

Objectives

Algorithms

Metrics

x

x

Outlook: Simulations

Outlook: Simulations

Outlook: Simulations

• Goal: A set of precalibrated ready-to-use large-scale agent-based transport simulation instances in France (eqasim / MATSim)

Questions?

https://slides.com/sebastianhorl/heart22