Condemned to re-invent SQL

* poorly

HI, Good morning!

Michał Lowas-Rzechonek

C / C++ / PYTHON

mrzechonek

STORAGE

DEPLOY

API

UI

UX

BUSINESS

AM I A FULL STACK NINJA?

STORAGE

DEPLOY

API

UI

UX

BUSINESS

* not to scale

NO, I AM NOT

Relational is inconvenient

  • Data types? Pfft
  • I can do validation in  the app
  • Just give me my list already!
  • Migrations suck

... or is it?

SAFETY HARNESS MUST BE WORN

  • Your app is not the only client
  • "SELECT *" is not the only query
  • Modify in place instead of export/import

* does not apply to mysql

active record

  • One of design patterns
  • Tables map to classes
  • Columns map to attributes
  • Rows map to objects

Code, or didn't happen

  • Django is a web framework
  • It's rather convenient
  • And comes with an ORM

Code, or didn't happen

  • PostgreSQL is super awesome
  • Most of this stuff is "just SQL"
  • Please stop using MySQL

Breadcrumbs

Milky waySolar systemEarth

Breadcrumbs

id parent title
1 NULL Universe
2 1 Milky Way
3 2 Sol
4 3 Earth
WITH RECURSIVE breadcrumb AS (

  SELECT 0 as level, id, parent, name
   FROM section
   WHERE id = 4

 UNION ALL

  SELECT level + 1, s.id, s.parent, s.name
   FROM breadcrumb b
   JOIN section s ON b.parent = s.id

 )
 SELECT * FROM breadcrumb
 ORDER BY level DESC;

Pagination

A-C   D-G   H-J   K-P   Q-Z

Pagination

page from to
1 A C
2 D G
3 H J
...
WITH pages AS (

  SELECT
    (row_number() OVER (ORDER BY surname) - 1) / 10
      AS page,
    name
  FROM person

)
SELECT DISTINCT
    page,
    first_value(name) OVER pagination AS "from",
    last_value(name) OVER pagination AS "to"
FROM pages
WINDOW pagination AS (PARTITION BY page)
ORDER BY page;

window queries

Given a lecture schedule,
How do we plan construction work
on the university campus?

window queries

SELECT
    "building",
    date_trunc('month', "date") AS "month",
    count(*) AS "count"
FROM "lectures_lecture"
LEFT JOIN "lectures_room"
       ON "lectures_room"."id" = "room_id"
GROUP BY "building", "month"
# month count
B3 2015-10-01 413
C3 2015-06-01 375
​B4 ​2015-04-01 373
B3 2015-02-01 373
B3 2015-12-01 412
...

window queries

WITH "busy_months" AS (...)

SELECT DISTINCT ON ("building")
    "building",
    first_value("month") OVER "busy_months"
    last_value("month") OVER "busy_months"
FROM
    "busy_months"
WINDOW "busy_months" AS (
    PARTITION BY "building"
    ORDER BY "count"
    ROWS BETWEEN UNBOUNDED PRECEDING AND
                 UNBOUNDED FOLLOWING
);
# min max
A1 2015-04-01 2015-11-01
A2 2015-02-01 2015-08-01
A3 2015-02-01 2015-07-01
B1 2015-02-01 2015-08-01
B2 2015-02-01 2015-12-01
B3 2015-02-01 2015-09-01
B4 2015-09-01 2015-07-01
...

JSON

date message
2015-12-08 { ...
"uri": "/"
}
2015-10-09 { ...
"code": "404"
}
2015-10-09 { ...
"size": "1024"
}
...
SELECT
    message ->> 'uri'
FROM
    event
WHERE
    message ->> 'status' = '404';



-- go to 11
CREATE INDEX
    journal_event_message_response
ON journal_event((message ->> 'response'));

Custom functions

CREATE LANGUAGE plythonu;

CREATE FUNCTION polynomial_fit(y FLOAT[],
                               degree INTEGER)
    RETURNS FLOAT[] AS
$$
    import numpy
    x = numpy.arange(len(y))
    return numpy.polyfit(x, y, degree)
$$
LANGUAGE plpythonu IMMUTABLE;

... and much more

  • full text search
  • custom types
  • GIS

Disclaimer

  • not a silver bullet
  • know thy tools
  • KISS, not fancy

thank you

mrzechonek

Condemned to re-invent SQL @ PyCon PL 2015

By Michał Lowas-Rzechonek

Condemned to re-invent SQL @ PyCon PL 2015

A few tips on integrating fancy SQL queries with Django's ORM

  • 642