The 2022 Wrap-up

A bit of history

15 h

188 h

484 sub

35 videos

GCP Service Introductions

Introduction to BigQuery

Introduction to Cloud Composer

BigQuery

Delta Tracking via DML Merge

Partitioning Strategy

Analytical Functions & ROW_NUMBER()

Atomic Operations

Cost Optimisation

Fill data gap using LEAD

BI Engine

Federated Queries for Cloud SQL

BigQuery's Storage Read API

Cloud Datastore

Key & Index Design

Cloud Datastore TTL

System Design

ETL vs ELT and Transient Data Warehouse

Scheduling based Batch Ingestion

Event based Batch Ingestion

Run DBT workload on Cloud Composer 2

Run DBT jobs with Cloud Batch

PubSub Message Consumer on GKE Autopilot

Scale GKE Autopilot in seconds

Ingest Kafka events to Pub/Sub and BigQuery

BigQuery Scalable and cost effective
backup solution

Split mixed data using dataflow

PubSub BigQuery Subscription

Super Lightweight Real-time Ingestion Design

Event Driven BigQuery Consumer

Handle Late Arriving Data using Workflow

CI/CD

CI/CD with CloudBuild

Logging & Monitoring

Monitoring Deployment using Auditlog

Use SQL to query logs via Log Analytics

Security

Private Access with Cloud IAP

Service Account Impersonation

Manage Access to Internal Apps on
App Engine using IAP

Data Privacy

Exploring Automatic DLP for BigQuery

Leadership

Team Leadership in Technology