About me

Francois Dion

Chief Data Scientist

Dion Research LLC

fdion@dionresearch.com

linkedIn                @f_dion

github

About you

The Hitchhiker's Guide to the Open

Source Data Science Galaxy

Francois Dion

June 10, 2016

Download PDF

John Tukey

 "Far better an approximate answer to the right question, which is often vague, than an exact answer to the wrong question, which can always be made precise."

The Data Science Wheel

Why?

"ex-libris" part V: Visualization (LinkedIn)

Apps

• QGIS: Open source GIS
• Blender: 3d creation tools
  • BlenderGIS: Blender addons to make the bridge between Blender and geographic data
  • BlenderFlowmapTools: for creating 2D vector fields/flowmaps
• Jupyter Notebook: the data science platform

Databases

• Postgresql: flexible database engine
  • Postgis: GIS extensions for Postgresql
  • crankshaft: CartoDB Spatial Analysis
  • imposm: import OSM data in Postgresql

Python (Data Wrangling)

• Sqlalchemy: ORM and abstraction layer for database queries (over a dozen dialects of SQL)
  • Psycopg2: Postgresql driver
  • GeoAlchemy2: gis extensions
• GDAL/OGR: abstraction layer for raster and for vector formats (200+)
• fiona: another OGR API

• numpy/pandas: array / panel data (dataframes)
• NetCDF4: disk persistence format
• shapely: Manipulation & analysis of geometric objects
• geopandas: geographic pandas extensions
• dask-geopandas: scaling geopandas
• cartoframes: carto DF

Python (Feature Engineering)

• Geopy: Geocoding addresses (18 API)
• censusgeocode: Python wrapper for the US Census Geocoder
• ip2geotools: Geocoding IPs (14 API)
• pyproj: Performs cartographic transformations and geodetic computations.

• cartopy: cartographic python library with matplotlib support
• scipy.spatial: Spatial algorithms and data structures (kdtree, voronoi, minkowski...)
• scikit-image: image processing, feature engineering

Waldo Tobler

 "everything is related to everything else, but near things are more related than distant things."

Python (Model)

• scikit-learn: machine learning, data mining
  • sklearn.cluster
  • sklearn.distance
• networkx: complex graphs and networks

• osmnx: OSM+networkx

• prophet: time series
• PySal: Python Spatial Library (also esda, giddy, spaghetti)

• PyTorch: Deep learning platform
• rpy2: interface with R packages

ex:

rpostgis, dplyr, tidyr, sp, shapefiles, raster, geojson, geosphere, leafletR, cartography, choroplethR, amdai, ggmap, caret, glm, forecast, deepboost

Python (Visualization)

• matplotlib and mpl_toolkits.basemap: old reliable (descartes)
• ipyleaflet: Jupyter notebook widget for interactive maps
• Folium: (branca)
• pptk: point processing toolkit 
• OSMNX: Open Street Maps + NetworkX

• Altair: visualization tools with Vega (altair, pdvega, altair_widgets, altair_matplotlib)
• PyViz: visualization tools with Bokeh (hvPlot, DataShader, EarthSim, GeoViews)
• Dash: visualization tools with Plotly (cufflinks, scattergeo, mapbox, figure_factory)

Thank you!

fdion@dionresearch.com

https://slides.com/fdion