ABOUT ME: Amélie Anglade

• DS and Music Information Retrieval Consultant
   
• PhD in MIR from QMUL
   
• Worked for Large R&D Labs: Sony CSL, Philips Research
   
• Now works mostly with music startups: SoundCloud, MusicGraph

ME and GRAPHS

• MSc. thesis on modelling and clustering P2P networks of listeners

ME and GRAPHS

• MSc. thesis on modelling and clustering P2P networks of listeners
   
• Designed and implemented the DiscoRank at SoundCloud

ME and GRAPHS

• MSc. thesis on modelling and clustering P2P networks of listeners
   
• Designed and implemented the DiscoRank at SoundCloud
   
• Worked for MusicGraph on Big Data graph algorithms

HOW to reach me

• @amelie on dsr07.slack.com 
   
• @utstikkar on Twitter (and the web)
   
• amelie.anglade@gmail.com

Outline

• Introduction
   
• Graph Theory and Algorithms
   
• Network properties and models
   
• PageRank Algorithm
   
• Graph Computing Technologies
   
• Programming project

intro

real-world graphs

World Wide Web

real-world graphs

Facebook: Open Graph & Graph Search

real-world graphs

graph theory and algorithms

network properties and models

one famous Graph algorithm: Pagerank

Graph computing tech

• In-memory graph toolkits
  • Usually single-user systems
  • Graph analysis and visualisation
  • Implementation of many if not all graph/network algorithms
  • Limit: can only operate on graphs that can be stored in main memory = max millions of edges
  • Examples: JUNG, NetworkX, iGraph

Graph computing tech

• Real-time graph databases

  • ​Design to support multi-user concurrency

  • Use disk to persist the graph = couple billion edges locally, hundreds of billions of edges on distributed systems

  • Limit: global graphs algorithms/analytics not feasible

  • Focus on local algorithms and traversals

  • Examples: Neo4j, OrientDB, InfiniteGraph, DEX, Titan

Graph computing tech

• Batch processing graph frameworks

  • ​Optimised for global graph analysis

  • Often Hadoop for storage (HDFS) and processing (MapReduce)

  • Iterative algorithms

  • Limit: Not real-time computation

  • But: Can leverage sequential reads from disk

  • Do not support concurrent users

  • Examples: Hama, Giraph, GraphLab, Faunus

TIME to play

with graph algorithms

• Environment: GraphLab
   

• Data: Stanford Large Network Dataset Collection