Marc Compte
GIS lover. Web developer. Working at the GIS Service in the University of Girona (http://udg.edu/sigte) and freelance.
The Mosquito Alert map implementation
A citizen science use case
GIS Service (SIGTE), University of Girona
marc.compte@udg.edu
https://slides.com/mcompte/mosquitomap/live
Presentation index
- The project
- The map front-end
- The map back-end
The project
Mosquito Alert is a cooperative citizen science observatory coordinated by different public research institutions.
What
Its main objective is to fight against the tiger mosquito and the yellow fever mosquito invasive spreading.
Why
How
Data flow chart
How
The software used in the Mosquito Alert application is free and open source. It is distributed under a license that allows you to use, change, improve the software and redistribute it.
Software licensing
Quadruple helix innovation*
*Martin Curley and Bror Salmelin, Open Innovation 2.0: A New Paradigm
Spheres involved II
Who
Who
Planning, decision-making, risk evaluation, awareness, pest control...
Vector risk predictions, epidemiological modelling, mitigation modelling
Pest control
Data collection and re-usage, knoweldge transfer, awareness
Spheres involved II
Who
Catalan Institution for Research and Advanced Studies
Centre for Advanced Studies.
Spanish Research Council (CEAB-CSIC)
Center for Ecological Research and Forestry Applications
Project coordinators
Figures
Citizens reporting
| 2017 | 2014 - 2017 | |
|---|---|---|
| App downloads | 7,415 | 45,005 |
| Tiger reports | 1,760 | 10,038 |
| YF reports | 3 | 6 |
| Breeding reports | 991 | 2,354 |
Experts validation
| Tiger confirmed | |
|---|---|
| 2017 | 42% |
| 2016 | 40% |
| 2015 | 40% |
| 2014 | 16% |
Scientific impact
| 2017 | |
|---|---|
| Articles | 7 |
| Quoted | 2 |
| Conferences | 23 |
| Awards | 2 |
Dissemination
| 2017 | |
|---|---|
| Events | 10 |
| Schools | 13 |
| Press/TV | 127 |
Internet
| 2017 | |
|---|---|
| 1,426 | |
| 4,341 | |
| Web visits | 25,675 |
| Newsletter | 218 |
Highlights
Source: Palmer et al. (2017). Citizen science provides a reliable and scalable tool
to track disease-carrying mosquitoes.
Nature Communications
, 8:916.
Barcelona City Awards 2017
Highlights
Source: Eritja et al. (2017). Hitchhiking mosquitoes: Direct Evidence of Adult Aedes albopictus Dispersal by Car.
Scientific Reports, 10:1038.
5 out of every 1,000 daily car trips in the Metropolitan Area of Barcelona during the summer carries a tiger mosquito.
The map front-end
The map front-end
Software stack
Categories of validated observations
Distribution of citizens reporting observations
Filtering by time, hashtag or municipality
Public features
The map front-end
Download
Metadata, help
Share this view
Printable reports
Public features
The map front-end
Public features
The map front-end
Print reports
Validation result
Picture
Expert's note
Metadata
Public features
The map front-end
Private features
The map front-end
- Validation probability
New observations menu, includes:
- Pending validation
- Folding feature
- Other categories
Private features
The map front-end
Custom layers
Private features
The map front-end
New filters
- Notification filters
Private features
The map front-end
New features
- Additional data downloaded
- Send notifications to app users
The map back-end
The map back-end
Kick-off & developers
project = start_project(
name='Mosquito Alert', date_start = 'March 2016')
project.enrole_developer(
name=u'Toni Hernández', python_experience=None, date='March 2016')
project.enrole_developer(
name=u'Àlex Busquets', python_experience='Short', date='March 2016')
project.quit_developer(
name=u'Àlex Busquets', date='September 2016')
project.enrole_developer(
name=u'Marc Compte', python_experience=None, date='September 2016')
The map back-end
Requirements
-
Django==2.0.3
User management, admin interface, routing, models, ... -
django-import-export==1.0.0
Import/export spread-sheets or csv files -
Pillow==5.0.0
ImageFile fields -
psycopg2-binary==2.7.4
PostgreSQL connector -
pyproj==1.9.5.1
Proj4 python port -
requests==2.18.4
Manage HTTP/1.1 requests
The map back-end
Non requirements
-
Fiona, Shapely, GeoPandas, PySal, ...
We do very little geo analysis (in PostGIS) -
GeoDjango
Our only geometries are points stored in float lat/lon fields -
Django REST Framework
We don't need a full CRUD REST API
The map back-end
Developer tools
-
GIT
Versioning -
python runserver
Development server -
Atom
Code/text editor. Basic packages:-
autocomplete-python
Autocomplete methods, functions, ... Show docstrings. -
linter-pylama
Linter for python. We share the same style-guide (PEP8+PEP257). -
python-indent
Auto indent code -
atom-ide-ui
Diagnostics, outline, ...
-
autocomplete-python
The map back-end
Back-end?
what back-end?
Focus on the front-end
The map back-end
Learning while doing
Started the project with:
- Focus on the front-end (+ little python knowledge)
- Lack of style-guide
- Little use of models
- Little modularity (+ duplication by necessity)
- No naming conventions (imperative vs- declarative)
At mid-project we realized:
- Code was getting bigger (more endpoints, models, ...)
- Styling and naming was not consistent
- Need to DRY the WET code
The map back-end
Django key files
-
urls.py
Routing -
views.py
Endpoint functions -
models.py
Database abstraction
| Simple metrics | first commit | pre refactoring | last commits |
|---|---|---|---|
| urls | 4 (~20 lines) | 12 (~30 lines) | 18 (~200 lines) |
| views | 4 (~200 lines) | 14 (~900 lines) | 15 (
|
| models | 1 (~30 lines) | 6 (~125 lines) | 13 (~400 lines) |
The map back-end
Lack of styleguide
Indentations - To read better and faster
Docstrings - To document
Use common packages for lint, auto-indent, ...
The map back-end
Lack of styleguide
Docstrings - To document
Use common packages for lint, auto-indent, ...
The map back-end
Lack of styleguide
Line length
Use common packages for lint, auto-indent, ...
The map back-end
Little use of models
Too many raw SQLs to manage
The map back-end
Little use of models
Some SQL extras
The map back-end
Little modularity
Need to DRY the WET code
The map back-end
Little modularity
Some WET urls.py code
The map back-end
Little modularity
DRYed urls.py code
The map back-end
Naming conventions
Make them more descriptive/declarative
Thank you!
Questions?
The Mosquito Alert map implementation A citizen science use case GIS Service (SIGTE), University of Girona marc.compte@udg.edu https://slides.com/mcompte/mosquitomap/live
Mosquito Alert Map API
By Marc Compte
Mosquito Alert Map API
How did we implement the API feeding the Mosquito Alert Map.
- 6,268
