Processing the
Social Web
Jezeniel Zapanta
Software Engineer - LenddoEFL
About Us
@jezeniel
Howard Lince III
Director of Engineering - LenddoEFL
Howard3
Agenda
What will we build?
Flask Crash Course
Celery
Activity
What will we build?
Setup
Requirements
- Bring your own laptop
- Python 3.6+
- Redis
- virtualenv
- curl or postman
- git
http://links.lenddo.com/python364
http://links.lenddo.com/pythonosx
Windows
Mac OS X
http://links.lenddo.com/pythonlinux
Linux
$ git clone https://github.com/jezeniel/processing-the-social-web
$ cd processing-the-social-web
$ python3 -m venv venv # if `python3` not found try `python`
$ source venv/bin/activate # OSX/Linux
$ venv/Scripts/activate # Windows
$ pip install -r requirements.txt
github.com/jezeniel/processing-the-social-web
What is Flask?
- It is a micro web framework for Python.
Use Cases
- Traditional web applications
- Microservices
- HTTP APIs
from flask import Flask, jsonify, request, render_template
app = Flask(__name__)
@app.route('/page/<name>/')
def page(name):
return render_template('hello.html', name=name)
@app.route('/api/get/<id>')
def index(id):
return jsonify({'id': id})
@app.route('/api/post/', methods=['POST'])
def api():
data = request.get_json()
return jsonify(request_body=data)
Crash Course
app.py
<html>
<body>
<h1>Hello, {{name}}</h1>
</body>
</html>
template/hello.html
Running Flask
$ export FLASK_APP=app.py
$ flask run
Linux/Mac
> set FLASK_APP=app.py
> flask run
Windows
from flask import Flask, jsonify, request, render_template
app = Flask(__name__)
@app.route('/page/<name>/')
def page(name):
return render_template('hello.html', name=name)
@app.route('/api/get/')
def index():
return jsonify({'hello': 'world'})
@app.route('/api/post/', methods=['POST'])
def api():
data = request.get_json()
response = {'request_body': data}
return jsonify(response)
Crash Course
app.py
<html>
<body>
<h1>Hello, {{name}}</h1>
</body>
</html>
template/hello.html
What is Celery?
- Async task/job queue for Python.
- Supports different brokers like Redis and RabbitMQ.
Architecture
Use Cases
- Background Processing
- Scheduled Periodic Work
- Priority Tasks
- Processing the Social Web (ha!)
from celery import Celery
from flask import Flask, jsonify, request, render_template
def make_celery(app):
celery = Celery(app.import_name,
broker=app.config['CELERY_BROKER_URL'],
backend=app.config['CELERY_RESULT_BACKEND'])
celery.conf.update(app.config)
TaskBase = celery.Task
class ContextTask(TaskBase):
abstract = True
def __call__(self, *args, **kwargs):
with app.app_context():
return TaskBase.__call__(self, *args, **kwargs)
celery.Task = ContextTask
return celery
app = Flask(__name__)
app.config.update(
CELERY_BROKER_URL='redis://localhost:6379',
CELERY_RESULT_BACKEND='redis://localhost:6379'
)
celery = make_celery(app)
@celery.task()
def add(a, b):
print(a + b)
return a + b
Flask Integration
@celery.task()
def add(a, b):
return a + b
...
@app.route('/api/add/<a>/<b>/')
def add_api(a, b):
result = add.delay(a, b)
return jsonify({'result': result.get()})
Executing Tasks
app.py
Remember?
$ celery -A app.celery worker
Running Workers
terminal
Then try again :)
How about this?
app.py
import time
...
@celery.task()
def add(a, b):
time.sleep(5)
print(a + b)
return a + b
@app.route('/api/add/<int:a>/<int:b>/')
def add_api(a, b):
result = add.delay(a, b)
return jsonify({'result': result.get()})
Will block the request until task is complete
from celery.result import AsyncResult
...
celery = make_celery(app)
...
@app.route('/api/add/<int:a>/<int:b>/')
def add_api(a, b):
result = add.delay(a, b)
return jsonify({'task_id': result.id})
@app.route('/api/results/<task_id>')
def results_api(task_id):
result = AsyncResult(id=task_id, app=celery)
return jsonify({'status': result.status, 'result': result.result})
Better Approach
Questions?
Activity
Python Packages
- facebook-sdk
- flask-sqlalchemy
- requests
from flask import Flask
from flask_sqlalchemy import SQLAlchemy
app = Flask(__name__)
app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:////tmp/test.db'
db = SQLAlchemy(app)
class User(db.Model):
id = db.Column(db.Integer, primary_key=True)
username = db.Column(db.String(80), unique=True, nullable=False)
email = db.Column(db.String(120), unique=True, nullable=False)
Flask SQLAlchemy
app.py
Basic usage
from app import db, User
db.create_all() # Create your db first if it doesn't exit
# Create an entry
guest = User(username='juan', email='juan.lazy@gmail.com')
db.session.add(guest)
db.session.commit()
# Querying
User.query.all()
User.query.filter_by(username='juan').first()
https://developers.facebook.com
Let's create a facebook app
http://links.lenddo.com/cognitive
Integrating Microsoft's Vision API
Questions?
Thank you!
Processing the Social Web
By Jezeniel Zapanta
Processing the Social Web
- 576