Title Text

#TheStreetsOfWomen

Jessica Sena

@jsenag
github.com/jessisena

Computer Engineer Posgraduate UX

#TheStreetsOfWomen

http://geochicasosm.github.io/lascallesdelasmujeres

Technical Process

2. Process data
https://blog.mapbox.com/mapping-female-versus-male-street-names-b4654c1e00d5
3. Generate the map
1. Extract data

1. Extract data

GOAL:
To create a file with ONLY the STREETS of a city

Tools:
Tile Calculator - ​OSM QA Tiles - TileReduce - Overpass API

1. Extract data

http://osmlab.github.io/osm-qa-tiles/

http://tools.geofabrik.de/

1. Extract data

Implements MapReduce

(parallel computing)

 

Distributed spatial analysis

 

JavaScript + Mbtiles

https://github.com/mapbox/tile-reduce

1. Extract data

Manages tiles
Manages partial results + the final one
"GEO" version of 
"MapReduce"

reduce.js

Exects operations and analisys

map.js

Tile

resultt

1. Extract data

BBOX definition and MBTILES path

reduce.js

const opts = {
    bbox: [-84.12, 9.9, -84.02, 9.96],
    log: true,
    zoom: 12,
    sources: [
      {
        name: 'osm',
        mbtiles: path.join(__dirname, 'data/latest.planet.mbtiles')      
      }
    ],
    maxWorkers: 4,
    map: __dirname+'/map.js'
};
.on('reduce', function(result, tile){
  num++;
  finalGeojson.features = finalGeojson.features.concat(result.features) ;
})

.on('end', function(error){
    (...)
});
Generate the final Geojson including only streets

1. Extract data

module.exports = function(tileLayers, tile, writeData, done){

  const osmRoads = cleanGeoms(normalize(flatten(clip(tileLayers.osm.osm, tile))));
  done(null, osmRoads);

};
STEP 1:    Clip    +      flatten + normalize

Turf.js

+

@mapbox/Tilebelt

map.js

geojson-flatten

+

geojson-normalize

1. Extract data

turfjs.org

PASO 2: CleanGeoms
LineString
MultiLineString 
Polygon

?

module.exports = function(tileLayers, tile, writeData, done){

  const osmRoads = cleanGeoms(normalize(flatten(clip(tileLayers.osm.osm, tile))));
  done(null, osmRoads);

};

map.js

1. Extract data

PASO 2: CleanGeoms
Polygon
module.exports = function(tileLayers, tile, writeData, done){

  const osmRoads = cleanGeoms(normalize(flatten(clip(tileLayers.osm.osm, tile))));
  done(null, osmRoads);

};

map.js

FIXED!

function isValidSquare(geom) {

  return (
    geom.geometry.type === 'Polygon' &&
    geom.properties.highway &&
    geom.properties.highway === 'pedestrian'&&
    geom.properties.area &&
    geom.properties.area === 'yes'
    && geom.properties.name !== undefined
  );

}

1. Extract data

STEP 2: CleanGeoms
module.exports = function(tileLayers, tile, writeData, done){

  var osmRoads = cleanGeoms(normalize(flatten(clip(tileLayers.osm.osm, tile))));
  done(null, osmRoads);

};

map.js

1. Extract data

PASO 2: CleanGeoms
 geom.properties = {
   name: geom.properties.name,
   id: geom.properties['@id'],
   wikipedia_link: '',
   gender: 'unknown',
   scale: ''
 };
module.exports = function(tileLayers, tile, writeData, done){

  var osmRoads = cleanGeoms(normalize(flatten(clip(tileLayers.osm.osm, tile))));
  done(null, osmRoads);

};

map.js

1. Extract data

SOLUTION: 
​CLIP!

Streets out

of boundaries

1. Extract data

CLIP:

https://www.npmjs.com/package/query-overpass

OSM ID

Geojson Boundary 
flatten 
+ 
booleanContains
+

1. Extract data

Streets

out  inside boundaries

1. Extract data

"Clean" name for the classification

¿How?

1. Extract data

const filterList = ['Paseo','Río', 'Avenida', 'Hacienda', 'Puerto', 'Callejón', 'Calle', 'Calzada', 'Camino', 'Av.','Paso', 'Cañada', 'Minas', 'Cerrada', 
    'Puebla', 'Principal', 'Central','Primera', 'Segunda', 'Portón', 'Lateral', 'Calz.', 'Corrido', 'Casa', 'Villa', 'Mejía', 
    'Vía', 'Via', 'Real', 'Isla', 'Avendida', 'Marisma', 'Rada', 'Raudal', 'Ribera', 'Embocadura', 'Cataratas', 'Médanos', 
    'Mirador', 'Av', 'Jardín',  'A.', 'Circuito','Gral.', 'Rincón', 'Calz', 'Rinconada', 'Periférico', 'Cda', 'Jardin', 
    'C.', 'Callejon', 'Colegio', 'Valle', 'avenida', 'camino', 'calle', 'Calle', 'Rotonda', 'Parqueo', 'Parque', 'entrada', 
    'Entrada', 'sendero', 'Sendero', 'Pasaje', 'pasaje', 'Puerto', 'Ciudad', 'Puente', 'Boulevard', 'Agrosuperior', 'Bodegas', 
    'Autobanco', 'SkyTrace', 'Plaza', 'Motel', 'C/', 'Rotonda', 'Drive', 'Residencial', 'Automac', 
    'Auto', 'Transcersal', 'Inter', 'Pasillo', 'Centro', 'Caminito', 'Arandas', 'Proveedores', 'Cajero', 'Zona', 'Primer', 'Res.'
  ];

Dictionary

of street prefixes

languages!

1. Extract data

1. Extract data

1. Extract data

FINAL RESULT:

  • nombreciudad_streets.geojson
  • list.csv

2. Manage the data

GOALS:

  • Streets classification FEMALE/MALE
  • Associate Wikipedia links
  • Manual review

SOLUTION 1:

2. Manage the data

2.1. Classification F/M

Use limitations

Slow (1request/name)

Parse "Name + Surname"

API NameSor
API

SOLUTION 2:

CSV list (source INE)

+25.000 classified names

Local 
Dictionary
github.com/marcboquet/spanish-names

2. Manage the data

2.1. Classification F/M

Load the dictionaries in memory

Take the "clean name" column in CSV 

Jacint (Verdaguer)

Does it appear in the women's dictionary?

Does it appear in the men's dictionary?

NO
YES
is a WOMEN
YES
is a MAN
NO
Discard the street
new Set();

2. Manage the data

SOLUTION 1:

2.2. Wikipedia Links

Slow (request/street)

Complex results parse

MediaWiki API

2. Manage the data

SOLUTION 2:

Wikidata
Wikiquote
Wikinoticas
Wikisource

68.000 resources

2. Manage the data

2.2. Wikipedia Links

68.000 resources

https://fusejs.io/

2. Manage the data

2.2. Wikipedia Links

SOLUTION 2:

2.3. Manual check

Streets named after a person

F/M Classification

Wikipedia links

2. Manage the data

nombreciudad_streets.geojson

How?

Generate FINAL GEOSJON

2. Manage the data

const streeMap = new Map();

(...)

streeMap.set(splitLine[0], {
    url: url,
    gender: splitLine[4],
    scale: ""
});
var finalGeojson = {
    "type": "FeatureCollection",
    "features": []
};

for (var key in geojson.features) {

 if(streeMap.has(geojson.features[key].properties.name)){        
   (...)    
    finalGeojson.features.push(geojson.features[key]);
 }
}

2. Manage the data

Generate FINAL GEOSJON

Generata

STATS

if(!tratadosList.has(geojson.features[key].properties.name)){

  tratadosList.add(geojson.features[key].properties.name);

  if (objValues.url !== ('' && null && undefined) && objValues.gender.toLowerCase() === FEMALE){
    stats.numLink++;
    stats.numFemale++;
  }else if (objValues.gender.toLowerCase() === FEMALE){
    stats.numFemale++;
    stats.numNoLink++;                 
    noLinkList += `\n${geojson.features[key].properties.name}`;                                                             
  }else{
    stats.numMale++;
  }                    
}

2. Manage the data

FINAL GEOSJON  Generated!

2. Manage the data

2. Manage the data*

(*known errors)

  • Filtering by HIGHWAY
  • Filtering by LINESTRING (squares)
  • 1street <> 1name
  • Incomplete DB of names
  • Classification criteria

3. Visualize the data

3. Visualize the data

    this.map = new mapboxgl.Map({
      container: 'map',
      style,
      center: initCenter, // [lng, lat]
      zoom: 1,
    });
Create map object (MyMap.js)
    map.addLayer({
      id: `${sourcename}-line`,
      type: 'line',
      source: {
        type: 'geojson',
        data: geojson,
      },
      layout: {
        'line-join': 'round',
        'line-cap': 'round',
      },
      paint: {
        'line-blur': ['case', ['==', ['get', 'wikipedia_link'], ''], 4, 1],
        'line-color': [
          'case',
          ['==', ['get', 'gender'], FEMALE],
          '#ffca3a',
          '#00B99E',
        ],
        'line-width': [
          'case',
          ['==', ['get', 'gender'], FEMALE],
          widthFemale,
          widthMale,
        ],
      },
    });
Add line layer:

3. Visualize the data

    map.addLayer({
      id: `${sourcename}-fill`,
      type: 'fill',
      source: {
        type: 'geojson',
        data: geojson,
      },
      layout: {},
      paint: {
        'fill-color': ['case', ['==', ['get', 'gender'], FEMALE], '#ffca3a', '#00B99E'],
        'fill-opacity': ['case', ['==', ['get', 'wikipedia_link'], ''], 0.2, 0.6],
      },
      filter: ['==', '$type', 'Polygon'],
    });
Add polygon layer:

3. Visualize the data

How to add a new city?

​

  1. Copy scripts results in data
     
  2. Add city in src/Constants.js
Constants.js
Scripts results files

3. Visualize the data

So far...

In the most of the cities, the ratio woman-man is around 15%-85%

Collaborative Project

github.com/geochicasosm/

Intrucciones en README.md

Still a lot of room

for improvement!

4. Opportunities

Prepare the project to accept other languages

  • Dictionary of streets prefixes
  • Dictionary of names
  • Wikipedia links
  • ...

4. Opportunities

Improve de criteria to remove or keep streets:

4. Opportunities

Define a women's taxonomy:

  • Which women do we have in our streets? (religious, scientists, artists?)
     
  • How is the classification used in Wikidata? Can we use it?
     
  • Check the one used in Barcelona

    *Add the data to the map!

4. Opportunities

Analyze the streets assignation:

  • In which  streets are the women assigned? main ones, secondary ones? new neighborhoods?
     
  • Why this happens? Is it a new trend?
     
  • Define street's taxonomy or use the one in OSM?

4. Opportunities

Revert data:

  • Prepare an automated way to revert the data in OSM

 

  • Prepare an automated way to revert the data in Wikidata

4. Opportunities

How to properly scale the project:

  • Visulization level (improve the menu, how to show small areas...)

 

  • Data loading level (vector tiles?)

4. Opportunities

Improve data analysis:

  • Prepare external page with more charts? (comparisson between cities, categories, ways...) 

 

  • What else could we extrapolate from the data?

4. Opportunities

Find a better way to find the wikipedia links:

  • The current process has a lot of errors, improve it or create a new one. 

 

4. Opportunities

Define a template for a social campaign to change the street names:

  • What steps should we need to follow
  • Who should we need to contact
  • How would we organize it and how many people would we need?
  • What else do would we need in the project (data related)
  • ...

 

4. Opportunities

PUT YOUR IDEA HERE
https://geochicasosm.github.io/lascallesdelasmujeres/
https://github.com/geochicasosm

@geochicasosm

www.geochicas.org

Made with Slides.com