An Introduction to BigQuery
Table of content
- Overview
- Usage
- Integration for machines
- Visualization for people
- Solution
- Community
BigQuery: Overview
- Query on Google Cloud
- Store massive data
-
Analysis in real time
(up to 2 minutes delay)
BigQuery: Usage
- query with SQL syntax, JSON result
- import/export CSV/JSON
- access control
- integration via RESTful API
Usage: Query
- table wildcard
- big join
- big group aggregration
- view (saved query) as subquery
Usage: import/export
- define table (denormalization)
- real time import: 100k record/sec
- parallel export (to n files)
Usage: access control
- dataset role --> view/update data
- project role --> run query, manage project
Usage: integration
- REST API (SDK)
- Google Spreadsheet
- Google App Script
- any web app
- Command line tool (bq)
- Connector for Excel
BigQuery: Scripting
- scripting with Google App Script
- supported by Google (scalability, authentication)
- always online (ready to be used by people & machines)
- connected with Google apps & APIs (drive, sheet, doc, form, mail, contact, map...)
- can use external JavaScript/CSS/API (with some restrictrions): jQuery, jQuery UI
BigQuery: Integration
- ETL: Talend Studio, SQLstream
- Analysis: tableau, Bime
- BI: Jaspersoft, Metric Insights
- more info:
https://cloud.google.com/bigquery/third-party-tools
BigQuery: Integration
tableau
Bime Analytics
BigQuery: visualization
- show trends and patterns in data
-
learn ideas from info-graphics & analytic systems (Google Analytics ...)
- switch layouts quickly (chart, graph, map ...)
- can combine many queries into a dashboard
- can change parameters on-the-fly
BigQuery+Map
airport delay correlations
Build a solution
- Dogfood: experiment with our team/organization first
- Define which data to capture (the more the better): events, transactions, people, relations...
- Build queries for their products, projects, departments, processes, employees, customers...
- Integrate output into their workflow (dashboard, report, email, notification ...)