The Mosquito Alert map implementation

A citizen science use case

GIS Service (SIGTE), University of Girona

marc.compte@udg.edu

https://slides.com/mcompte/mosquitomap/live

Presentation index

  1. The project
  1. The map front-end
  1. The map back-end

The project

Mosquito Alert is a cooperative citizen science observatory coordinated by different public research institutions.

What

Its main objective is to fight against the tiger mosquito and the yellow fever mosquito invasive spreading.

Why

How

Data flow chart

How

The software used in the Mosquito Alert application is free and open source. It is distributed under a license that allows you to use, change, improve the software and redistribute it.

Software licensing

Quadruple helix innovation*

*Martin Curley and Bror Salmelin, Open Innovation 2.0: A New Paradigm

Spheres involved II

Who

Who

Planning, decision-making, risk evaluation,  awareness, pest control...

Vector risk predictions, epidemiological modelling, mitigation modelling

Pest control

Data collection and re-usage, knoweldge transfer,  awareness

Spheres involved II

Who

Catalan Institution for Research and Advanced Studies

Centre for Advanced Studies.

Spanish Research Council (CEAB-CSIC)

Center for Ecological Research and Forestry Applications

Project coordinators

Figures

Citizens reporting

2017 2014 - 2017
App downloads 7,415 45,005
Tiger reports 1,760 10,038
YF reports 3 6
Breeding reports 991 2,354

Experts validation

Tiger confirmed
2017 42%
2016 40%
2015 40%
2014 16%

Scientific impact

2017
Articles 7
Quoted 2
Conferences 23
Awards 2

Dissemination

2017
Events 10
Schools 13
Press/TV 127

Internet

2017
Twitter 1,426
Facebook 4,341
Web visits 25,675
Newsletter 218

Highlights

Source: Palmer et al. (2017). Citizen science provides a reliable and scalable tool

to track disease-carrying mosquitoes.

Nature Communications

, 8:916.

Barcelona City Awards 2017

Highlights

Source: Eritja et al. (2017). Hitchhiking mosquitoes: Direct Evidence of Adult Aedes albopictus Dispersal by Car.

Scientific Reports, 10:1038.

5 out of every 1,000 daily car trips in the Metropolitan Area of Barcelona during the summer carries a tiger mosquito.

The map front-end

The map front-end

Software stack

Categories of validated observations

Distribution of citizens reporting observations

Filtering by time, hashtag or municipality

Public features

The map front-end

Download

Metadata, help

Share this view

Printable reports

Public features

The map front-end

Public features

The map front-end

Print reports

Validation result

Picture

Expert's note

Metadata

Public features

The map front-end

Private features

The map front-end

  • Validation probability

New observations menu, includes:

  • Pending validation
  • Folding feature
  • Other categories

Private features

The map front-end

Custom layers

Private features

The map front-end

New filters

  • Notification filters

Private features

The map front-end

New features

  • Additional data downloaded
  • Send notifications to app users

The map back-end

The map back-end

Kick-off & developers

project = start_project(

    name='Mosquito Alert', date_start = 'March 2016')

project.enrole_developer(

    name=u'Toni Hernández', python_experience=None, date='March 2016')

 

project.enrole_developer(

    name=u'Àlex Busquets', python_experience='Short', date='March 2016')

project.quit_developer(

    name=u'Àlex Busquets', date='September 2016')

project.enrole_developer(

    name=u'Marc Compte', python_experience=None, date='September 2016')

The map back-end

Requirements

  • Django==2.0.3
    User management, admin interface, routing, models, ...
  • django-import-export==1.0.0
    Import/export spread-sheets or csv files
  • Pillow==5.0.0
    ImageFile fields
  • psycopg2-binary==2.7.4
    PostgreSQL connector
  • pyproj==1.9.5.1
    Proj4 python port
  • requests==2.18.4
    Manage HTTP/1.1 requests

The map back-end

Non requirements

  • Fiona, Shapely, GeoPandas, PySal, ...
    We do very little geo analysis (in PostGIS)
  • GeoDjango
    Our only geometries are points stored in float lat/lon fields
  • Django REST Framework
    We don't need a full CRUD REST API

The map back-end

Developer tools

  • GIT
    Versioning
  • python runserver
    Development server
  • Atom
    Code/text editor. Basic packages:
    • autocomplete-python
      Autocomplete methods, functions, ... Show docstrings.
    • linter-pylama
      Linter for python. We share the same style-guide (PEP8+PEP257).
    • python-indent
      Auto indent code
    • atom-ide-ui
      Diagnostics, outline, ...

The map back-end

Back-end?

what back-end?

Focus on the front-end

The map back-end

Learning while doing

Started the project with:

  • Focus on the front-end (+ little python knowledge)
  • Lack of style-guide
  • Little use of models
  • Little modularity (+ duplication by necessity)
  • No naming conventions (imperative vs- declarative)

At mid-project we realized:

  • Code was getting bigger (more endpoints, models, ...)
  • Styling and naming was not consistent
  • Need to DRY the WET code

The map back-end

Django key files

  • urls.py
    Routing
  • views.py
    Endpoint functions
  • models.py
    Database abstraction
Simple metrics first commit pre refactoring last commits
urls 4 (~20 lines) 12 (~30 lines) 18 (~200 lines)
views 4 (~200 lines) 14 (~900 lines) 15 ( ~300 lines)
models 1 (~30 lines) 6 (~125 lines) 13 (~400 lines)

The map back-end

Lack of styleguide

Indentations - To read better and faster

Docstrings - To document

Use common packages for lint, auto-indent, ...

The map back-end

Lack of styleguide

Docstrings - To document

Use common packages for lint, auto-indent, ...

The map back-end

Lack of styleguide

Line length

Use common packages for lint, auto-indent, ...

The map back-end

Little use of models

Too many raw SQLs to manage

The map back-end

Little use of models

Some SQL extras

The map back-end

Little modularity

Need to DRY the WET code

The map back-end

Little modularity

Some WET urls.py code

The map back-end

Little modularity

DRYed urls.py code

The map back-end

Naming conventions

Make them more descriptive/declarative

Thank you!

Questions?

GIS Service (SIGTE), University of Girona

marc.compte@udg.edu

@marcompte

Mosquito Alert Map API

By Marc Compte

Mosquito Alert Map API

How did we implement the API feeding the Mosquito Alert Map.

  • 6,071