An Introduction to BigQuery

Table of content

  • Overview
  • Usage
  • Integration       for machines
  • Visualization    for people
  • Solution
  • Community

BigQuery: Overview

  • Query on Google Cloud
  • Store massive data
  • Analysis in real time
    (up to 2 minutes delay)

BigQuery: Usage

  • query with SQL syntax, JSON result
  • import/export CSV/JSON
  • access control
  • integration via RESTful API

Usage: Query

  • table wildcard
  • big join
  • big group aggregration
  • view (saved query) as subquery

Usage: import/export

  • define table (denormalization)
  • real time import: 100k record/sec
  • parallel export (to n files)

Usage: access control

  • dataset role --> view/update data
  • project role --> run query, manage project

Usage: integration

  • REST API (SDK)
    • Google Spreadsheet
    • Google App Script
    • any web app
  • Command line tool (bq)
  • Connector for Excel

BigQuery: Scripting

  • scripting with Google App Script
  • supported by Google (scalability, authentication)
  • always online (ready to be used by people & machines)
  • connected with Google apps & APIs (drive, sheet, doc, form, mail, contact, map...)
  • can use external JavaScript/CSS/API (with some restrictrions): jQuery, jQuery UI

BigQuery: Integration

BigQuery: Integration

tableau

Bime Analytics

BigQuery: visualization

  • show trends and patterns in data
  • learn ideas from info-graphics & analytic systems (Google Analytics ...)
    • switch layouts quickly (chart, graph, map ...)
    • can combine many queries into a dashboard
    • can change parameters on-the-fly

BigQuery+Map

airport delay correlations

Build a solution

  • Dogfood: experiment with our team/organization first
  • Define which data to capture (the more the better): events, transactions, people, relations...
  • Build queries for their products, projects, departments, processes, employees, customers...
  • Integrate output into their workflow (dashboard, report, email, notification ...)

Community

More info

An Introduction to BigQuery

By Djung

Made with Slides.com

An Introduction to BigQuery

What can you do with BigQuery?

  • 401