Synthetic population workshop

Synthetic population workshop

Introduction

Sebastian Hörl

26 October 2023

IRT SystemX

Today's workshop

Population synthesis using the eqasim toolkit is gaining interest
- Transport (obviously)
- Energy
- Epidemics
- ...
Various institutions and developers / researchers working with it
- Which components can be contributed?
- How can we make that easier?
- What is everybody's roadmap?
- What are opportunities for collaboration?
(Today's focus is not the Java / transport simulation part!)

Users & Implementations

eqasim-org/ile-de-france

Users & Implementations

eqasim-org/ile-de-france

Contributions

Repository

Commits

eqasim-org/ile-de-france

Repository

Recent CHANGELOG

Major recent changes

Updated all input data to latest versions (INSEE 2019, ...)

Major recent changes

Updated all input data to latest versions (INSEE 2019, ...)
Integration of BD-TOPO buildings and BAN for home locations (Arthur)

Major recent changes

Updated all input data to latest versions (INSEE 2019, ...)
Integration of BD-TOPO buildings and BAN for home locations (Arthur)
Easier management of input files
- Archives (zip, 7z, ...) stay packaged while reading
- Automatic discovery of files in directories (BD-TOPO, GTFS, ...)

Contributing

Make a fork & clone it

git clone git@github.com:user/ile-de-france.git

Contributing

Make a fork & clone it
Make a branch

git branch my_changes

Contributing

Make a fork & clone it
Make a branch
Integrate your changes & commit & push
- Ideally, make use of Conventional Commits to name the commits

git commit -m "feat: new great feature"

git push origin my_changes

Contributing

Make a fork & clone it
Make a branch
Integrate your changes & commit & push
Send a Pull Request (PR) on Github

Contributing

Make a fork & clone it
Make a branch
Integrate your changes & commit & push
Send a Pull Request (PR) on Github
Continuous testing will check the code

Contributing

Make a fork & clone it
Make a branch
Integrate your changes & commit & push
Send a Pull Request (PR) on Github
Continuous testing will check the code
Maintainers will review the changes before merging them in

Extensions

New functionality in pipeline: Aliasing
Allows to replace stages via configuration

Extensions

New functionality in pipeline: Aliasing
Allows to replace stages via configuration

Extensions

New functionality in pipeline: Aliasing
Allows to replace stages via configuration

Extensions

New functionality in pipeline: Aliasing
Allows to replace stages via configuration

Agenda

10h30 - 12h00: Presentation of recent developments
12h00 - 13h00: Lunch and exchange
13h00 - 14h00: Presentation of recent developments
14h00 - 15h00: Short presentation of research agendas
15h00 - 16h00: Roadmap and discussion

Synthetic population workshop

IRT SystemX Roadmap

Specific developments

Contributing matching by unité urbaine
Flexible processing of GTFS data with different time frames (Tarek)
Automated configuration of the pipeline (using a boundary as a basis)
Distinction between synthesis-based trips (only origin and destination) and choice-based trips (mode, travel time, ...)

November 2023

December 2023

Before end of 2024

Research topics

Benchmarking population synthesis
- Unified implementation of approaches to compare performance
Population scenarios
- Population growth, with different scenarios, ...
Location choice
- Workplace: take into account CSP, ...
- Secondary: shop surface, attraction, ...
Interfacing of the pipeline
- With 4-step-models, other ABMs like SUMO, ...

Open Demand Project

Currently, large confusion between the two major components of eqasim

Demand synthesis

pipeline

Standardized MATSim

with DCM

Open Demand Project

Currently, large confusion between the two major components of eqasim

Demand synthesis

pipeline

Standardized MATSim

with DCM

We propose to rename the synthesis pipeline to

Open Demand Project

Currently, large confusion between the two major components of eqasim

Demand synthesis

pipeline

Standardized MATSim

with DCM

We propose to rename the synthesis pipeline to

Open Demand Project

Initially, no major changes, just a new name

Open Demand Project

In the midterm restructuring of the project is planned
Currently, we have multiple individual cases with a lot of code duplication, not very modular

Open Demand Project

In the midterm restructuring of the project is planned
Step 1: Establish a common library of demand synthesis algorithms and wrappers around existing code and repositories (via dependencies)

Open Demand Project

In the midterm restructuring of the project is planned
Step 1: Establish a common library of demand synthesis algorithms and wrappers around existing code and repositories (via dependencies)

import opendemand as odm
df_weights = odm.ipf(df_persons, df_marginals)

df_matches = odm.statistical_matching(...)

Open Demand Project

In the midterm restructuring of the project is planned
Step 1: Establish a common library of demand synthesis algorithms and wrappers around existing code and repositories (via dependencies)

python3 -m odmp config_nantes.yml

Open Demand Project

In the midterm restructuring of the project is planned
Step 1: Establish a common library of demand synthesis algorithms and wrappers around existing code and repositories (via dependencies)
Step 2: Replace custom pipeline by established tools such as Snakemake

Open Demand Project

In the midterm restructuring of the project is planned
Step 1: Establish a common library of demand synthesis algorithms and wrappers around existing code and repositories (via dependencies)
Step 2: Replace custom pipeline by established tools such as Snakemake

rule step1:
    params:
        text="some text"
    output:
        "step1_output.txt"
    script:
        "step1.py"

rule step2:
    input:
        "step1_output.txt"
    output:
        "step2_output.txt"
    script:
        "step2.py"

snakemake pipeline.smk