Processing the

Social Web

Jezeniel Zapanta

Software Engineer - LenddoEFL

About Us

@jezeniel

Howard Lince III

Director of Engineering - LenddoEFL

Howard3

Agenda

What will we build?

Flask Crash Course

Celery

Activity

What will we build?

Setup

Requirements

  • Bring your own laptop
  • Python 3.6+
  • Redis
  • virtualenv
  • curl or postman
  • git
http://links.lenddo.com/python364
http://links.lenddo.com/pythonosx

Windows

Mac OS X

http://links.lenddo.com/pythonlinux

Linux

$ git clone https://github.com/jezeniel/processing-the-social-web
$ cd processing-the-social-web

$ python3 -m venv venv # if `python3` not found try `python`


$ source venv/bin/activate  # OSX/Linux
$ venv/Scripts/activate     # Windows

$ pip install -r requirements.txt
github.com/jezeniel/processing-the-social-web

What is Flask?

  • It is a micro web framework for Python.

Use Cases

  • Traditional web applications
  • Microservices
  • HTTP APIs
from flask import Flask, jsonify, request, render_template

app = Flask(__name__)


@app.route('/page/<name>/')
def page(name):
    return render_template('hello.html', name=name)


@app.route('/api/get/<id>')
def index(id):
    return jsonify({'id': id})


@app.route('/api/post/', methods=['POST'])
def api():
    data = request.get_json()
    return jsonify(request_body=data)

Crash Course

app.py
<html>
    <body>
        <h1>Hello, {{name}}</h1>
    </body>
</html>
template/hello.html

Running Flask

$ export FLASK_APP=app.py
$ flask run

Linux/Mac

> set FLASK_APP=app.py
> flask run

Windows

from flask import Flask, jsonify, request, render_template

app = Flask(__name__)


@app.route('/page/<name>/')
def page(name):
    return render_template('hello.html', name=name)


@app.route('/api/get/')
def index():
    return jsonify({'hello': 'world'})


@app.route('/api/post/', methods=['POST'])
def api():
    data = request.get_json()
    response = {'request_body': data}
    return jsonify(response)

Crash Course

app.py
<html>
    <body>
        <h1>Hello, {{name}}</h1>
    </body>
</html>
template/hello.html

What is Celery?

  • Async task/job queue for Python.
  • Supports different brokers like Redis and RabbitMQ.

Architecture

Use Cases

  • Background Processing
  • Scheduled Periodic Work
  • Priority Tasks
  • Processing the Social Web (ha!)
from celery import Celery
from flask import Flask, jsonify, request, render_template

def make_celery(app):
    celery = Celery(app.import_name,
                    broker=app.config['CELERY_BROKER_URL'],
                    backend=app.config['CELERY_RESULT_BACKEND'])
    celery.conf.update(app.config)
    TaskBase = celery.Task
    class ContextTask(TaskBase):
        abstract = True
        def __call__(self, *args, **kwargs):
            with app.app_context():
                return TaskBase.__call__(self, *args, **kwargs)
    celery.Task = ContextTask
    return celery


app = Flask(__name__)
app.config.update(
    CELERY_BROKER_URL='redis://localhost:6379',
    CELERY_RESULT_BACKEND='redis://localhost:6379'
)
celery = make_celery(app)


@celery.task()
def add(a, b):
    print(a + b)
    return a + b

Flask Integration

@celery.task()
def add(a, b):
    return a + b

...

@app.route('/api/add/<a>/<b>/')
def add_api(a, b):
    result = add.delay(a, b)
    return jsonify({'result': result.get()})
    

Executing Tasks

app.py

Remember?

$ celery -A app.celery worker

Running Workers

terminal

Then try again :)

How about this?

app.py
import time

...

@celery.task()
def add(a, b):
    time.sleep(5)
    print(a + b)
    return a + b
@app.route('/api/add/<int:a>/<int:b>/')
def add_api(a, b):
    result = add.delay(a, b)
    return jsonify({'result': result.get()})
    

Will block the request until task is complete

from celery.result import AsyncResult

...

celery = make_celery(app)

...

@app.route('/api/add/<int:a>/<int:b>/')
def add_api(a, b):
    result = add.delay(a, b)
    return jsonify({'task_id': result.id})


@app.route('/api/results/<task_id>')
def results_api(task_id):
    result = AsyncResult(id=task_id, app=celery)
    return jsonify({'status': result.status, 'result': result.result})

Better Approach

Questions?

Activity

Python Packages

  • facebook-sdk
  • flask-sqlalchemy
  • requests
from flask import Flask
from flask_sqlalchemy import SQLAlchemy

app = Flask(__name__)
app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:////tmp/test.db'
db = SQLAlchemy(app)


class User(db.Model):
    id = db.Column(db.Integer, primary_key=True)
    username = db.Column(db.String(80), unique=True, nullable=False)
    email = db.Column(db.String(120), unique=True, nullable=False)

Flask SQLAlchemy

app.py

Basic usage

from app import db, User
db.create_all() # Create your db first if it doesn't exit

# Create an entry
guest = User(username='juan', email='juan.lazy@gmail.com')
db.session.add(guest)
db.session.commit()

# Querying
User.query.all()
User.query.filter_by(username='juan').first()
https://developers.facebook.com

Let's create a facebook app

http://links.lenddo.com/cognitive

Integrating Microsoft's Vision API

Questions?

Thank you!

Processing the Social Web

By Jezeniel Zapanta

Processing the Social Web

  • 576