Bazel Introduction

@JiaLiPassion at NGCHINA 2020

Who am I

  • Name: Jia Li
  • Company: @ThisDot
  • Zone.js: Code Owner
  • Angular: Collaborator
  • @JiaLiPassion

Bazel

  • Bazel introduction
  • Bazel in TypeScript project
  • Bazel in Angular
  • Demo for mono repo project

What is Bazel

  • A Build Tool
  • Incremental
  • For any languages/Frameworks
  • Scalable

Bazel is a replacement of Jenkins or Webpack?

  • CI: Jenkins/CircleCI/...
  • tsc/rollup/webpack/sass/...
  • make/grunt/gulp/...

AlexEagle at ngconf 2019

Bazel is Hub

AlexEagle at ngconf 2019

Why not just continue to use gulp

The goal for Build tool

  • Essential:
    • Correct - don't need to worry about environment
    • Fast
      • Incremental
      • Parallelism
    • Predictable - same input -> same output
    • Reusable -> build rule/task can be composed and reused
  • Nice to have:
    • Universal -> support multiple languages/framework

Correctness

  • Bazel use sandbox to isolate the environment
  • Sandbox support Linux and Mac (Some features not supported)

Universal

  • Bazel doesn't rely on any specified framework or languages
  • Bazel is a perfect build tool for full stack development inside mono repo.

Gulp is imperative

gulp.task('build', () => {
  return gulp.src(['src/*.ts'])
             .pipe(tsc(...))
             .pipe(gulp.dest('../build/'));
})
gulp.task('test', ['build'], () => {
  ...
});
  1. gulp will run build task
  2. gulp will run test task

 

Bazel is declarative

Sample

// date.ts
export function formatDate(d: Date) {
  return `${d.getFullYear()}-${d.getMonth()}-${d.getDate()}!!`;
}

// user.ts
import { formatDate } from './date';

/** Data object representing a User. */
export class User {
  constructor(readonly name: string, readonly birthday: Date) {}

  toString() {
    return `${this.name}, born on ${formatDate(this.birthday)}`;
  }
}

// birthday_card.ts
import { User } from './user';

/** Prints a birthday greeting for the given user to the console. */
export function printBirthdayGreeting(user: User) {
  console.log(`Happy birthday ${user.name}, ${user.birthday} is your day!`);
}

Dependencies

date.ts

user.ts

birthday_card.ts

Gulp is imperative

date.ts task

user.ts task

birthday_card.ts
task

Bazel is declarative

date.ts target

user.ts target

birthday_card.ts
target

Bazel target

# date typescript build target
ts_library(
    name = "date",
    srcs = ["date.ts"],
)

date.ts target

input

date.ts

ts_library rule

date

output

1. date.d.ts

2. date.js

Bazel target

# date typescript build target
ts_library(
    name = "date",
    srcs = ["date.ts"],
)

# user typescript build target
ts_library(
    name = "user",
    srcs = ["user.ts"],
    deps = [":date"]
)

date target

user rule

input

date.ts

ts_library rule

date

output

1. date.d.ts

2. date.js

input

user.ts

input

date.d.ts

ts_library rule

user

output

1. user.d.ts

2. user.js

Demo

Predictive

result = f(input)

Bazel dependency graph

Bazel

  • Incremental - Analyze and only run really impacted targets

  • Deterministic - We can cache build results because rules are pure function

  • Hermetic - remote execution, parallelization

  • Composable - Bazel plugins are like Unix pipes

  • Universal - Builds Android, iOS, web, backends, cloud services, and more (in theory everything)

  • Industrial grade - All google big mono repo projects are using Babel

Concept and Terminology

  • Workspace
  • Package
  • Label
  • Rule
  • Build File

Structure

WORKSPACE

Package

Target

Workspace

  • Directory in the file system contains
    • source files of your project
    • symbolic links that contains your build output 
  • workspace definition is in a file named WORKSPACE at the root of the directory.

WORKSPACE

workspace(
    name = "com_china_2020",
    managed_directories = {"@npm": ["node_modules"]},
)

load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")

http_archive(
    name = "build_bazel_rules_nodejs",
    sha256 = "0942d188f4d0de6ddb743b9f6642a26ce1ad89f09c0035a9a5ca5ba9615c96aa",
    urls = ["https://github.com/bazelbuild/rules_nodejs/releases/download/0.38.1/rules_nodejs-0.38.1.tar.gz"],
)

1. workspace name should be unique globally. Use something like reverse dns name such as (com_ngchina_2020)

2. in WORKSPACE file, we should do

    - install env such as yarn/npm/bazel

    - setup toolchain such typescript

Packages

  • The primary unit of code organization(something like module) in a repository
  • Collection of related files and a specification of the dependencies among them
  • Directory containing a file named BUILD or BUILD.bazel, residing beneath the top-level directory in the workspace
  • A package includes all files in its directory, plus all subdirectories beneath it, except those which themselves contain a BUILD file

Packages

Gulp structure

gulp.task('build-animations', () => {});

gulp.task('build-core', () => {});

gulp.task('build-core-schematics', () => {});

...

gulp.task('test-animations', () => {});

1. gulp file doesn't have 1:1 relationship to module directory.

2. gulp can reference files from any directories in gulp task.

Bazel Packages

# WORKSPACE file is at the root of directory

├── WORKSPACE
├── BUILD.bazel

├── ...
├── animations
│   ├── BUILD.bazel
│   ├── ...
│   ├── browser
│   │   └── BUILD.bazel
 |     |   ...
│   ├── test
│   │   ├── BUILD.bazel
│   │   └── ...

 

# root package 

# animations package 

# animations/browser package 

# animations/test package 

BUILD file

Starlark language

def fizz_buzz(n):
  """Print Fizz Buzz numbers"""
  for i in range(1, n + 1):
    s = ""
    if i % 3 == 0:
      s += "Fizz"
    if i % 5 == 0:
      s += "Buzz"
    print(s if s else i)

fizz_buzz(20)

1. Subset of Python

2. class, import, while, yield, lambda, is, raise, etc not supported

3. Recursion not allowed

4. Most of python builtin methods not supported

BUILD file

  • contains build targets
  • build targets can be 
    • files
    • rules
  • target can depend on other target
    • circular dependency are not allowed
    • two target generate same output will cause problem

BUILD file

package(default_visibility = ["//visibility:private"])

load("@npm_bazel_typescript//:index.bzl", "ts_library")

ts_library(
    name = "lib",
    srcs = [":lib.ts"],
    visibility = ["//visibility:public"],
)

target

package(default_visibility = ["//visibility:private"])

load("@npm_bazel_typescript//:index.bzl", "ts_library")

ts_library(
    name = "lib",
    srcs = [":lib.ts"],
    visibility = ["//visibility:public"],
)

Buildifier

yarn add -D @bazel/buildifier

"scripts": {
        "bazel:format": "find . -type f \\( -name \"*.bzl\" -or -name WORKSPACE -or -name BUILD -or -name BUILD.bazel \\) ! -path \"*/node_modules/*\" | xargs buildifier -v --warnings=attr-cfg,attr-license,attr-non-empty,attr-output-default,attr-single-file,constant-glob,ctx-actions,ctx-args,depset-iteration,depset-union,dict-concatenation,duplicated-name,filetype,git-repository,http-archive,integer-division,load,load-on-top,native-build,native-package,out-of-order-load,output-group,package-name,package-on-top,positional-args,redefined-variable,repository-name,same-origin-load,string-iteration,unsorted-dict-items,unused-variable",
        "bazel:lint": "yarn bazel:format --lint=warn",
        "bazel:lint-fix": "yarn bazel:format --lint=fix"
}

package.json

Bazel Query

Bazel Query

  • Bazel can statically analyze BUILD.bazel files
  • Using Bazel Query to understand the project build structure (can output graph)
  • Bazel query can also return results for further operations. 

Sample Queries

# All the dependencies of //src/app
bazel query 'deps(//src/app)'
# All the targets in //src
bazel query '//src:*'

# Path between any of the targets in src/app/... and any in //src/lib
bazel query 'somepath(foo/..., //bar/baz:all)'

# Reverse dependencies of //src/lib within the transitive closure
# of the universe set //src/app
bazel query 'rdeps(//src/app, //src/lib:date)'

Bazel query Visualize

# Show graphical representation of the build graph
bazel query --output=graph ... | dot -Tpng > graph.png

need to install graphviz

Target

Build Target

  • Files/Rules Target
    • Files such as tsconfig.json
    • Rules such as ts_library()
  • Target input/output are known at build time
  • Target will only be built when input changed.

bazel build

bazel build

bazel build //src/app

bazel build output

bazel info
  1. bazel-bin: build files
  2. output_path: all output artifacts during the process

Bundle with rollup

yarn add -D @bazel/rollup rollup
load("@build_bazel_rules_nodejs//:defs.bzl", "rollup_bundle")


rollup_bundle(
    name = "app_bundle",
    entry_point = "//src/app:birthday_card.ts",
    deps = ["//src/app"],
)

bazel build //src:app_bundle

watch mode

yarn add -D @bazel/ibazel

ts_devserver

ts_devserver(
    name = "devserver",
    index_html = "index.html",
    port = 4200,
    deps = [
        ":app_bundle.es2015.js",
    ],
)

jasmine_node_test rule

load("@npm_bazel_jasmine//:index.bzl", "jasmine_node_test")
load("@npm_bazel_typescript//:index.bzl", "ts_library")

ts_library(
    name = "test_lib",
    testonly = True,
    srcs = [":date.spec.ts"],
    deps = [
        "//src/lib:date",
        "@npm//@types/jasmine",
    ],
)

jasmine_node_test(
    name = "test",
    deps = [":test_lib"],
)

bazel test

bazel test //src/...

bazel test //src/lib/test

Karma rule

load("@npm_bazel_karma//:index.bzl", "karma_web_test_suite")

ts_library(
    name = "test_lib",
    testonly = True,
    srcs = [":date.spec.ts"],
    deps = [
        "//src/lib:date",
        "@npm//@types/jasmine",
    ],
)

karma_web_test_suite(
    name = "test_web",
    deps = [":test_lib"],
)

Protractor rule

load("@npm_bazel_protractor//:index.bzl", "protractor_web_test_suite")
load("@npm_bazel_typescript//:index.bzl", "ts_library")

ts_library(
    name = "e2e_lib",
    testonly = True,
    srcs = [":app.spec.ts"],
    deps = [
        "@npm//@types/jasmine",
        "@npm//jasmine",
        "@npm//protractor",
    ],
)

protractor_web_test_suite(
    name = "e2e",
    on_prepare = ":protractor.on-prepare.js",
    server = "//src:devserver",
    deps = [":e2e_lib"],
)

Other common rules

Terser

yarn add -D @bazel/terser terser

load("@npm_bazel_terser//:index.bzl", "terser_minified")

terser_minified(
    name = "app_bundle_min",
    src = ":app_bundle.es2015.js",
)

genrule

genrule(
    name = "copy",
    srcs = ["//somepackage:sometarget"],
    outs = ["output.js"],
    cmd = "cp $< $@",
)

npm_package

npm_package(
    name = "npm_package",
    srcs = [
        "CHANGELOG.md",
        "README.md",
        "package.json",
    ],
    visibility = ["//visibility:public"],
    deps = [
        ":LICENSE.wrapped",
        ":LICENSE_copy",
        "//packages/app",
    ],
)

Angular with Bazel

Use @angular/bazel

# For existing project
ng add @angular/bazel

# For new project
npm install -g @angular/bazel
ng new --collection=@angular/bazel

Use @angular-devkit/architect-cli

load("@npm//@angular-devkit/architect-cli:index.bzl", "architect")

architect(
    name = "build",
    args = [
        "frontend:build",
        "--outputPath=$(@D)",
    ],
    ...
)

Difference

tsc

@angular/compiler-cli 

tsc-wrapped

@angular-devkit/architect

angular bazel

architect

angular_ts_library

Full stack demo

Troubleshooting

  • Check bazel-out folder
  • Environment issues
    • bazel clean
    • bazel clean --expunge
    • bazel shutdown

Thank you

References

  • bazel presentation by Martin Probst at Angular connect 2018
  • bazel introduction by Alex Eagle at ng-conf 2019
  • bazel training by Minko
  • https://angular.io/guide/bazel
  • https://github.com/JiaLiPassion/ngchina2020-bazel
  • @JiaLiPassion
Made with Slides.com