Bazel Introduction

@JiaLiPassion at NGCHINA 2020

Who am I

Name: Jia Li
Company: @ThisDot
Zone.js: Code Owner
Angular: Collaborator
@JiaLiPassion

Bazel

Bazel introduction
Bazel in TypeScript project
Bazel in Angular
Demo for mono repo project

What is Bazel

A Build Tool
Incremental
For any languages/Frameworks
Scalable

Bazel is a replacement of Jenkins or Webpack?

CI: Jenkins/CircleCI/...
tsc/rollup/webpack/sass/...
make/grunt/gulp/...

AlexEagle at ngconf 2019

Bazel is Hub

AlexEagle at ngconf 2019

Why not just continue to use gulp

The goal for Build tool

Essential:
- Correct - don't need to worry about environment
- Fast
  - Incremental
  - Parallelism
- Predictable - same input -> same output
- Reusable -> build rule/task can be composed and reused
Nice to have:
- Universal -> support multiple languages/framework

Correctness

Bazel use sandbox to isolate the environment
Sandbox support Linux and Mac (Some features not supported)

Universal

Bazel doesn't rely on any specified framework or languages
Bazel is a perfect build tool for full stack development inside mono repo.

Gulp is imperative

gulp.task('build', () => {
  return gulp.src(['src/*.ts'])
             .pipe(tsc(...))
             .pipe(gulp.dest('../build/'));
})
gulp.task('test', ['build'], () => {
  ...
});

gulp will run build task
gulp will run test task

Bazel is declarative

Sample

// date.ts
export function formatDate(d: Date) {
  return `${d.getFullYear()}-${d.getMonth()}-${d.getDate()}!!`;
}

// user.ts
import { formatDate } from './date';

/** Data object representing a User. */
export class User {
  constructor(readonly name: string, readonly birthday: Date) {}

  toString() {
    return `${this.name}, born on ${formatDate(this.birthday)}`;
  }
}

// birthday_card.ts
import { User } from './user';

/** Prints a birthday greeting for the given user to the console. */
export function printBirthdayGreeting(user: User) {
  console.log(`Happy birthday ${user.name}, ${user.birthday} is your day!`);
}

Dependencies

date.ts

user.ts

birthday_card.ts

Gulp is imperative

date.ts task

user.ts task

birthday_card.ts
task

Bazel is declarative

date.ts target

user.ts target

birthday_card.ts
target

Bazel target

# date typescript build target
ts_library(
    name = "date",
    srcs = ["date.ts"],
)

date.ts target

input

date.ts

ts_library rule

date

output

1. date.d.ts

2. date.js

Bazel target

# date typescript build target
ts_library(
    name = "date",
    srcs = ["date.ts"],
)

# user typescript build target
ts_library(
    name = "user",
    srcs = ["user.ts"],
    deps = [":date"]
)

date target

user rule

input

date.ts

ts_library rule

date

output

1. date.d.ts

2. date.js

input

user.ts

input

date.d.ts

ts_library rule

user

output

1. user.d.ts

2. user.js

Demo

Predictive

result = f(input)

Bazel dependency graph

Bazel

Incremental - Analyze and only run really impacted targets
Deterministic - We can cache build results because rules are pure function
Hermetic - remote execution, parallelization
Composable - Bazel plugins are like Unix pipes
Universal - Builds Android, iOS, web, backends, cloud services, and more (in theory everything)
Industrial grade - All google big mono repo projects are using Babel

Setup Bazel

npm install
Prebuilt binary download

npm install globally

$ npm i -g @bazel/bazel

Download preload binary

Install Bazel from https://bazel.build.

Windows needs to install MSYS2, WSL are not supported yet.

Concept and Terminology

Workspace
Package
Label
Rule
Build File

Structure

WORKSPACE

Package

Target

Workspace

Directory in the file system contains
- source files of your project
- symbolic links that contains your build output
workspace definition is in a file named WORKSPACE at the root of the directory.

WORKSPACE

workspace(
    name = "com_china_2020",
    managed_directories = {"@npm": ["node_modules"]},
)

load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")

http_archive(
    name = "build_bazel_rules_nodejs",
    sha256 = "0942d188f4d0de6ddb743b9f6642a26ce1ad89f09c0035a9a5ca5ba9615c96aa",
    urls = ["https://github.com/bazelbuild/rules_nodejs/releases/download/0.38.1/rules_nodejs-0.38.1.tar.gz"],
)

1. workspace name should be unique globally. Use something like reverse dns name such as (com_ngchina_2020)

2. in WORKSPACE file, we should do

- install env such as yarn/npm/bazel

- setup toolchain such typescript

Packages

The primary unit of code organization(something like module) in a repository
Collection of related files and a specification of the dependencies among them
Directory containing a file named BUILD or BUILD.bazel, residing beneath the top-level directory in the workspace
A package includes all files in its directory, plus all subdirectories beneath it, except those which themselves contain a BUILD file

Packages

Gulp structure

gulp.task('build-animations', () => {});

gulp.task('build-core', () => {});

gulp.task('build-core-schematics', () => {});

...

gulp.task('test-animations', () => {});

1. gulp file doesn't have 1:1 relationship to module directory.

2. gulp can reference files from any directories in gulp task.

Bazel Packages

# WORKSPACE file is at the root of directory

├── WORKSPACE
├── BUILD.bazel

├── ...
├── animations
│ ├── BUILD.bazel
│ ├── ...
│ ├── browser
│ │ └── BUILD.bazel
| | ...
│ ├── test
│ │ ├── BUILD.bazel
│ │ └── ...

# root package

# animations package

# animations/browser package

# animations/test package

BUILD file

Starlark language

def fizz_buzz(n):
  """Print Fizz Buzz numbers"""
  for i in range(1, n + 1):
    s = ""
    if i % 3 == 0:
      s += "Fizz"
    if i % 5 == 0:
      s += "Buzz"
    print(s if s else i)

fizz_buzz(20)

1. Subset of Python

2. class, import, while, yield, lambda, is, raise, etc not supported

3. Recursion not allowed

4. Most of python builtin methods not supported

BUILD file

contains build targets
build targets can be
- files
- rules
target can depend on other target
- circular dependency are not allowed
- two target generate same output will cause problem

BUILD file

package(default_visibility = ["//visibility:private"])

load("@npm_bazel_typescript//:index.bzl", "ts_library")

ts_library(
    name = "lib",
    srcs = [":lib.ts"],
    visibility = ["//visibility:public"],
)

Label: name of the target

@com_ngtaiwan_2019//lib:build

Workspace name

package

target

shorthand without WORKSPACE

//lib:build

package

target

shorthand without target

//lib

package

//lib:lib

package

target

package visibility

package(default_visibility = ["//visibility:private"])

load("@npm_bazel_typescript//:index.bzl", "ts_library")

ts_library(
    name = "lib",
    srcs = [":lib.ts"],
    visibility = ["//visibility:public"],
)

private
public
//some_package:__subpackages__

import

package(default_visibility = ["//visibility:private"])

load("@npm_bazel_typescript//:index.bzl", "ts_library")

ts_library(
    name = "lib",
    srcs = [":lib.ts"],
    visibility = ["//visibility:public"],
)

import rule
import macro

target

package(default_visibility = ["//visibility:private"])

load("@npm_bazel_typescript//:index.bzl", "ts_library")

ts_library(
    name = "lib",
    srcs = [":lib.ts"],
    visibility = ["//visibility:public"],
)

Buildifier

yarn add -D @bazel/buildifier

"scripts": {
        "bazel:format": "find . -type f \\( -name \"*.bzl\" -or -name WORKSPACE -or -name BUILD -or -name BUILD.bazel \\) ! -path \"*/node_modules/*\" | xargs buildifier -v --warnings=attr-cfg,attr-license,attr-non-empty,attr-output-default,attr-single-file,constant-glob,ctx-actions,ctx-args,depset-iteration,depset-union,dict-concatenation,duplicated-name,filetype,git-repository,http-archive,integer-division,load,load-on-top,native-build,native-package,out-of-order-load,output-group,package-name,package-on-top,positional-args,redefined-variable,repository-name,same-origin-load,string-iteration,unsorted-dict-items,unused-variable",
        "bazel:lint": "yarn bazel:format --lint=warn",
        "bazel:lint-fix": "yarn bazel:format --lint=fix"
}

package.json

Bazel Query

Bazel can statically analyze BUILD.bazel files
Using Bazel Query to understand the project build structure (can output graph)
Bazel query can also return results for further operations.

Sample Queries

# All the dependencies of //src/app
bazel query 'deps(//src/app)'
# All the targets in //src
bazel query '//src:*'

# Path between any of the targets in src/app/... and any in //src/lib
bazel query 'somepath(foo/..., //bar/baz:all)'

# Reverse dependencies of //src/lib within the transitive closure
# of the universe set //src/app
bazel query 'rdeps(//src/app, //src/lib:date)'

Query targets

# All the targets in //src/lib
bazel query '//src/lib:*'
# All rules in //src
bazel query '//src/lib:all'
# All targets under //src
bazel query '//src/...'

:* will match all targets (files and rules)
:all will only match all rules
... will include all sub packages.

Query dependencies

# query dependencies of //src/app
bazel query 'deps(//src/app)'
# why //src/app depends on //src/lib
bazel query 'somepath(//src/app, //src/lib)'
bazel query 'allpaths(//src/app, //src/lib)'
# query who depends on //src/lib
bazel query 'rdeps(..., //src/lib)'

somepath will return any possible path (only one)
allpaths will return all paths.
reverse dependency query may very expensive.

Tag

ts_library(
    name = "lib",
    srcs = [":index.ts"],
    tags = ["build-target"],
    visibility = ["//visibility:public"],
    deps = [
        ":date",
        ":user",
    ],
)

Query by tags

# query all targets with tags "build-target" under //src
bazel query --output=label 'attr("tags", "\[.*build-target.*\]", //src/...)'

complex query

# query all targets with tags "build-target" under //src and label must
# ends with lib
bazel query --output=label 'attr("tags", "\[.*build-target.*\]", //src/...) 
            intersect kind(".*lib", //src/...)'

intersect
except
union
...

Bazel query Visualize

# Show graphical representation of the build graph
bazel query --output=graph ... | dot -Tpng > graph.png

need to install graphviz

Target

Build Target

Files/Rules Target
- Files such as tsconfig.json
- Rules such as ts_library()
Target input/output are known at build time
Target will only be built when input changed.

Rule Target Naming

*_binary
- executable programs in a given language (nodejs_binary)
*_test
- special _binary rule for testing
*_library
- compiled module for given language (ts_library)

Rule Target

ts_library(
    name = "lib",
    srcs = [":index.ts"],
    tags = ["build-target"],
    visibility = ["//visibility:public"],
    deps = [
        ":date",
        ":user",
    ],
)

Common attributes

name - unique name within this package
srcs - inputs of the target, typically files
deps - compile-time dependencies
data - runtime dependencies
testonly - target which should be executed only when running bazel test
visibility - specifies who can make a dependency on the given target

Data: runtime dep

http_server(
   name = "prodserver",
   data = [
       "index.html",
       ":bundle",
       "styles.css",
   ],
)

data should only be used at runtime
data will not be analyzed at build time

bazel build

bazel build //src/app

bazel build output

bazel info

bazel-bin: build files
output_path: all output artifacts during the process

Bundle with rollup

yarn add -D @bazel/rollup rollup

load("@build_bazel_rules_nodejs//:defs.bzl", "rollup_bundle")


rollup_bundle(
    name = "app_bundle",
    entry_point = "//src/app:birthday_card.ts",
    deps = ["//src/app"],
)

bazel build //src:app_bundle

Request different formats rollup

bazel build //src:app_bundle.es2015.js
bazel build //src:app_bundle.umd.js
bazel build //src:app_bundle.min.umd.js

Dev Server

watch mode

yarn add -D @bazel/ibazel

ts_devserver

ts_devserver(
    name = "devserver",
    index_html = "index.html",
    port = 4200,
    deps = [
        ":app_bundle.es2015.js",
    ],
)

Jasmine test

jasmine_node_test

yarn add -D @bazel/jasmine jasmine @types/jasmine

jasmine_node_test rule

load("@npm_bazel_jasmine//:index.bzl", "jasmine_node_test")
load("@npm_bazel_typescript//:index.bzl", "ts_library")

ts_library(
    name = "test_lib",
    testonly = True,
    srcs = [":date.spec.ts"],
    deps = [
        "//src/lib:date",
        "@npm//@types/jasmine",
    ],
)

jasmine_node_test(
    name = "test",
    deps = [":test_lib"],
)

bazel test

bazel test //src/...

bazel test //src/lib/test

Debug jasmine test

test:debug --test_arg=--node_options=--inspect-brk --test_output=streamed --test_strategy=exclusive --test_timeout=9999 --nocache_test_results

.bazelrc

bazel test //src/lib/test --config=debug
// open chrome://inspect

Karma test

Karma setup

yarn add -D @bazel/karma karma

# Load karma dependencies
load("@npm_bazel_karma//:package.bzl",
     "npm_bazel_karma_dependencies")

npm_bazel_karma_dependencies()

# Set up web testing, choose browsers we can test on
load("@io_bazel_rules_webtesting//web:repositories.bzl",
     "browser_repositories", "web_test_repositories")

web_test_repositories()
browser_repositories(
    chromium = True,
)

Karma rule

load("@npm_bazel_karma//:index.bzl", "karma_web_test_suite")

ts_library(
    name = "test_lib",
    testonly = True,
    srcs = [":date.spec.ts"],
    deps = [
        "//src/lib:date",
        "@npm//@types/jasmine",
    ],
)

karma_web_test_suite(
    name = "test_web",
    deps = [":test_lib"],
)

run/debug karma

bazel test //src/lib/test:test_web
bazel run //src/lib/test:test_web

Protractor

Protractor setup

yarn add -D protractor @bazel/protractor

Protractor rule

load("@npm_bazel_protractor//:index.bzl", "protractor_web_test_suite")
load("@npm_bazel_typescript//:index.bzl", "ts_library")

ts_library(
    name = "e2e_lib",
    testonly = True,
    srcs = [":app.spec.ts"],
    deps = [
        "@npm//@types/jasmine",
        "@npm//jasmine",
        "@npm//protractor",
    ],
)

protractor_web_test_suite(
    name = "e2e",
    on_prepare = ":protractor.on-prepare.js",
    server = "//src:devserver",
    deps = [":e2e_lib"],
)

Other common rules

Terser

yarn add -D @bazel/terser terser

load("@npm_bazel_terser//:index.bzl", "terser_minified")

terser_minified(
    name = "app_bundle_min",
    src = ":app_bundle.es2015.js",
)

filegroup

filegroup(
    name = "test_env_scripts",
    srcs = [
        "..."
    ],
)

alias

alias(
    name = "tsconfig.json",
    actual = "//src:tsconfig.json",
    visibility = ["//visibility:public"],
)

genrule

genrule(
    name = "copy",
    srcs = ["//somepackage:sometarget"],
    outs = ["output.js"],
    cmd = "cp $< $@",
)

glob

sh_test(
    name = "mytest",
    srcs = ["mytest.sh"],
    data = glob(
        ["testdata/*.txt"],
        exclude = ["testdata/experimental.txt"],
    ),
)

npm_package

npm_package(
    name = "npm_package",
    srcs = [
        "CHANGELOG.md",
        "README.md",
        "package.json",
    ],
    visibility = ["//visibility:public"],
    deps = [
        ":LICENSE.wrapped",
        ":LICENSE_copy",
        "//packages/app",
    ],
)

Angular with Bazel

Use @angular/bazel

# For existing project
ng add @angular/bazel

# For new project
npm install -g @angular/bazel
ng new --collection=@angular/bazel

Use @angular-devkit/architect-cli

load("@npm//@angular-devkit/architect-cli:index.bzl", "architect")

architect(
    name = "build",
    args = [
        "frontend:build",
        "--outputPath=$(@D)",
    ],
    ...
)