Bazel Introduction

@JiaLiPassion at NGCHINA 2020

Who am I

  • Name: Jia Li
  • Company: @ThisDot
  • Zone.js: Code Owner
  • Angular: Collaborator
  • @JiaLiPassion

Bazel

  • Bazel introduction
  • Bazel in TypeScript project
  • Bazel in Angular
  • Demo for mono repo project

What is Bazel

  • A Build Tool
  • Incremental
  • For any languages/Frameworks
  • Scalable

Bazel is a replacement of Jenkins or Webpack?

  • CI: Jenkins/CircleCI/...
  • tsc/rollup/webpack/sass/...
  • make/grunt/gulp/...

AlexEagle at ngconf 2019

Bazel is Hub

AlexEagle at ngconf 2019

Why not just continue to use gulp

The goal for Build tool

  • Essential:
    • Correct - don't need to worry about environment
    • Fast
      • Incremental
      • Parallelism
    • Predictable - same input -> same output
    • Reusable -> build rule/task can be composed and reused
  • Nice to have:
    • Universal -> support multiple languages/framework

Correctness

  • Bazel use sandbox to isolate the environment
  • Sandbox support Linux and Mac (Some features not supported)

Universal

  • Bazel doesn't rely on any specified framework or languages
  • Bazel is a perfect build tool for full stack development inside mono repo.

Gulp is imperative

gulp.task('build', () => {
  return gulp.src(['src/*.ts'])
             .pipe(tsc(...))
             .pipe(gulp.dest('../build/'));
})
gulp.task('test', ['build'], () => {
  ...
});
  1. gulp will run build task
  2. gulp will run test task

 

Bazel is declarative

Sample

// date.ts
export function formatDate(d: Date) {
  return `${d.getFullYear()}-${d.getMonth()}-${d.getDate()}!!`;
}
// user.ts
import { formatDate } from './date';

/** Data object representing a User. */
export class User {
  constructor(readonly name: string, readonly birthday: Date) {}

  toString() {
    return `${this.name}, born on ${formatDate(this.birthday)}`;
  }
}
// birthday_card.ts
import { User } from './user';

/** Prints a birthday greeting for the given user to the console. */
export function printBirthdayGreeting(user: User) {
  console.log(`Happy birthday ${user.name}, ${user.birthday} is your day!`);
}

Dependencies

date.ts

user.ts

birthday_card.ts

Gulp is imperative

date.ts task

user.ts task

birthday_card.ts
task

Bazel is declarative

date.ts target

user.ts target

birthday_card.ts
target

Bazel target

# date typescript build target
ts_library(
    name = "date",
    srcs = ["date.ts"],
)

date.ts target

input

date.ts

ts_library rule

date

output

1. date.d.ts

2. date.js

Bazel target

# date typescript build target
ts_library(
    name = "date",
    srcs = ["date.ts"],
)
# user typescript build target
ts_library(
    name = "user",
    srcs = ["user.ts"],
    deps = [":date"]
)

date target

user rule

input

date.ts

ts_library rule

date

output

1. date.d.ts

2. date.js

input

user.ts

input

date.d.ts

ts_library rule

user

output

1. user.d.ts

2. user.js

Demo

Predictive

result = f(input)

Bazel dependency graph

Bazel

  • Incremental - Analyze and only run really impacted targets

  • Deterministic - We can cache build results because rules are pure function

  • Hermetic - remote execution, parallelization

  • Composable - Bazel plugins are like Unix pipes

  • Universal - Builds Android, iOS, web, backends, cloud services, and more (in theory everything)

  • Industrial grade - All google big mono repo projects are using Babel

Setup Bazel

  • npm install
  • Prebuilt binary download

npm install globally

$ npm i -g @bazel/bazel

Download preload binary

Install Bazel from https://bazel.build.

Windows needs to install MSYS2, WSL are not supported yet.

Concept and Terminology

  • Workspace
  • Package
  • Label
  • Rule
  • Build File

Structure

WORKSPACE

Package

Target

Workspace

  • Directory in the file system contains
    • source files of your project
    • symbolic links that contains your build output 
  • workspace definition is in a file named WORKSPACE at the root of the directory.

WORKSPACE

workspace(
    name = "com_china_2020",
    managed_directories = {"@npm": ["node_modules"]},
)

load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")

http_archive(
    name = "build_bazel_rules_nodejs",
    sha256 = "0942d188f4d0de6ddb743b9f6642a26ce1ad89f09c0035a9a5ca5ba9615c96aa",
    urls = ["https://github.com/bazelbuild/rules_nodejs/releases/download/0.38.1/rules_nodejs-0.38.1.tar.gz"],
)

1. workspace name should be unique globally. Use something like reverse dns name such as (com_ngchina_2020)

2. in WORKSPACE file, we should do

    - install env such as yarn/npm/bazel

    - setup toolchain such typescript

Packages

  • The primary unit of code organization(something like module) in a repository
  • Collection of related files and a specification of the dependencies among them
  • Directory containing a file named BUILD or BUILD.bazel, residing beneath the top-level directory in the workspace
  • A package includes all files in its directory, plus all subdirectories beneath it, except those which themselves contain a BUILD file

Packages

Gulp structure

gulp.task('build-animations', () => {});

gulp.task('build-core', () => {});

gulp.task('build-core-schematics', () => {});

...

gulp.task('test-animations', () => {});

1. gulp file doesn't have 1:1 relationship to module directory.

2. gulp can reference files from any directories in gulp task.

Bazel Packages

# WORKSPACE file is at the root of directory

├── WORKSPACE
├── BUILD.bazel

├── ...
├── animations
│   ├── BUILD.bazel
│   ├── ...
│   ├── browser
│   │   └── BUILD.bazel
 |     |   ...
│   ├── test
│   │   ├── BUILD.bazel
│   │   └── ...

 

# root package 

# animations package 

# animations/browser package 

# animations/test package 

BUILD file

Starlark language

def fizz_buzz(n):
  """Print Fizz Buzz numbers"""
  for i in range(1, n + 1):
    s = ""
    if i % 3 == 0:
      s += "Fizz"
    if i % 5 == 0:
      s += "Buzz"
    print(s if s else i)

fizz_buzz(20)

1. Subset of Python

2. class, import, while, yield, lambda, is, raise, etc not supported

3. Recursion not allowed

4. Most of python builtin methods not supported

BUILD file

  • contains build targets
  • build targets can be 
    • files
    • rules
  • target can depend on other target
    • circular dependency are not allowed
    • two target generate same output will cause problem

BUILD file

package(default_visibility = ["//visibility:private"])

load("@npm_bazel_typescript//:index.bzl", "ts_library")

ts_library(
    name = "lib",
    srcs = [":lib.ts"],
    visibility = ["//visibility:public"],
)

Label: name of the target

@com_ngtaiwan_2019//lib:build

Workspace name

package

target

shorthand without WORKSPACE 

//lib:build

package

target

shorthand without target 

//lib

package

//lib:lib

package

target

package visibility

package(default_visibility = ["//visibility:private"])

load("@npm_bazel_typescript//:index.bzl", "ts_library")

ts_library(
    name = "lib",
    srcs = [":lib.ts"],
    visibility = ["//visibility:public"],
)
  • private
  • public
  • //some_package:__subpackages__

import

package(default_visibility = ["//visibility:private"])

load("@npm_bazel_typescript//:index.bzl", "ts_library")

ts_library(
    name = "lib",
    srcs = [":lib.ts"],
    visibility = ["//visibility:public"],
)
  • import rule
  • import macro

target

package(default_visibility = ["//visibility:private"])

load("@npm_bazel_typescript//:index.bzl", "ts_library")

ts_library(
    name = "lib",
    srcs = [":lib.ts"],
    visibility = ["//visibility:public"],
)

Buildifier

yarn add -D @bazel/buildifier
"scripts": {
        "bazel:format": "find . -type f \\( -name \"*.bzl\" -or -name WORKSPACE -or -name BUILD -or -name BUILD.bazel \\) ! -path \"*/node_modules/*\" | xargs buildifier -v --warnings=attr-cfg,attr-license,attr-non-empty,attr-output-default,attr-single-file,constant-glob,ctx-actions,ctx-args,depset-iteration,depset-union,dict-concatenation,duplicated-name,filetype,git-repository,http-archive,integer-division,load,load-on-top,native-build,native-package,out-of-order-load,output-group,package-name,package-on-top,positional-args,redefined-variable,repository-name,same-origin-load,string-iteration,unsorted-dict-items,unused-variable",
        "bazel:lint": "yarn bazel:format --lint=warn",
        "bazel:lint-fix": "yarn bazel:format --lint=fix"
}

package.json

Bazel Query

Bazel Query

  • Bazel can statically analyze BUILD.bazel files
  • Using Bazel Query to understand the project build structure (can output graph)
  • Bazel query can also return results for further operations. 

Sample Queries

# All the dependencies of //src/app
bazel query 'deps(//src/app)'
# All the targets in //src
bazel query '//src:*'

# Path between any of the targets in src/app/... and any in //src/lib
bazel query 'somepath(foo/..., //bar/baz:all)'

# Reverse dependencies of //src/lib within the transitive closure
# of the universe set //src/app
bazel query 'rdeps(//src/app, //src/lib:date)'

Query targets

# All the targets in //src/lib
bazel query '//src/lib:*'
# All rules in //src
bazel query '//src/lib:all'
# All targets under //src
bazel query '//src/...'
  1. :* will match all targets (files and rules)
  2. :all will only match all rules
  3. ... will include all sub packages.

Query dependencies

# query dependencies of //src/app
bazel query 'deps(//src/app)'
# why //src/app depends on //src/lib
bazel query 'somepath(//src/app, //src/lib)'
bazel query 'allpaths(//src/app, //src/lib)'
# query who depends on //src/lib
bazel query 'rdeps(..., //src/lib)'
  1. somepath will return any possible path (only one) 
  2. allpaths will return all paths.
  3. reverse dependency query may very expensive.

Tag

ts_library(
    name = "lib",
    srcs = [":index.ts"],
    tags = ["build-target"],
    visibility = ["//visibility:public"],
    deps = [
        ":date",
        ":user",
    ],
)

Query by tags

# query all targets with tags "build-target" under //src
bazel query --output=label 'attr("tags", "\[.*build-target.*\]", //src/...)'

complex query

# query all targets with tags "build-target" under //src and label must
# ends with lib
bazel query --output=label 'attr("tags", "\[.*build-target.*\]", //src/...) 
            intersect kind(".*lib", //src/...)'
  1. intersect
  2. except
  3. union
  4. ...

Bazel query Visualize

# Show graphical representation of the build graph
bazel query --output=graph ... | dot -Tpng > graph.png

need to install graphviz

Target

Build Target

  • Files/Rules Target
    • Files such as tsconfig.json
    • Rules such as ts_library()
  • Target input/output are known at build time
  • Target will only be built when input changed.

Rule Target Naming

  • *_binary
    • executable programs in a given language (nodejs_binary) 
  • *_test
    • special _binary rule for testing
  • *_library
    • compiled module for given language (ts_library)

Rule Target

ts_library(
    name = "lib",
    srcs = [":index.ts"],
    tags = ["build-target"],
    visibility = ["//visibility:public"],
    deps = [
        ":date",
        ":user",
    ],
)

Common attributes

  • name - unique name within this package
  • srcs - inputs of the target, typically files
  • deps - compile-time dependencies
  • data - runtime dependencies
  • testonly - target which should be executed only when running bazel test
  • visibility - specifies who can make a dependency on the given target

Data: runtime dep

http_server(
   name = "prodserver",
   data = [
       "index.html",
       ":bundle",
       "styles.css",
   ],
)
  1. data should only be used at runtime
  2. data will not be analyzed at build time

bazel build

bazel build

bazel build //src/app

bazel build output

bazel info
  1. bazel-bin: build files
  2. output_path: all output artifacts during the process

Bundle with rollup

yarn add -D @bazel/rollup rollup
load("@build_bazel_rules_nodejs//:defs.bzl", "rollup_bundle")


rollup_bundle(
    name = "app_bundle",
    entry_point = "//src/app:birthday_card.ts",
    deps = ["//src/app"],
)
bazel build //src:app_bundle

Request different formats rollup

bazel build //src:app_bundle.es2015.js
bazel build //src:app_bundle.umd.js
bazel build //src:app_bundle.min.umd.js

Dev Server

watch mode

yarn add -D @bazel/ibazel

ts_devserver

ts_devserver(
    name = "devserver",
    index_html = "index.html",
    port = 4200,
    deps = [
        ":app_bundle.es2015.js",
    ],
)

Jasmine test

jasmine_node_test

yarn add -D @bazel/jasmine jasmine @types/jasmine

jasmine_node_test rule

load("@npm_bazel_jasmine//:index.bzl", "jasmine_node_test")
load("@npm_bazel_typescript//:index.bzl", "ts_library")

ts_library(
    name = "test_lib",
    testonly = True,
    srcs = [":date.spec.ts"],
    deps = [
        "//src/lib:date",
        "@npm//@types/jasmine",
    ],
)

jasmine_node_test(
    name = "test",
    deps = [":test_lib"],
)

bazel test

bazel test //src/...

bazel test //src/lib/test

Debug jasmine test

test:debug --test_arg=--node_options=--inspect-brk --test_output=streamed --test_strategy=exclusive --test_timeout=9999 --nocache_test_results

.bazelrc

bazel test //src/lib/test --config=debug
// open chrome://inspect

Karma test

Karma setup

yarn add -D @bazel/karma karma
# Load karma dependencies
load("@npm_bazel_karma//:package.bzl",
     "npm_bazel_karma_dependencies")

npm_bazel_karma_dependencies()

# Set up web testing, choose browsers we can test on
load("@io_bazel_rules_webtesting//web:repositories.bzl",
     "browser_repositories", "web_test_repositories")

web_test_repositories()
browser_repositories(
    chromium = True,
)

Karma rule

load("@npm_bazel_karma//:index.bzl", "karma_web_test_suite")

ts_library(
    name = "test_lib",
    testonly = True,
    srcs = [":date.spec.ts"],
    deps = [
        "//src/lib:date",
        "@npm//@types/jasmine",
    ],
)

karma_web_test_suite(
    name = "test_web",
    deps = [":test_lib"],
)

run/debug karma

bazel test //src/lib/test:test_web
bazel run //src/lib/test:test_web

Protractor

Protractor setup

yarn add -D protractor @bazel/protractor

Protractor rule

load("@npm_bazel_protractor//:index.bzl", "protractor_web_test_suite")
load("@npm_bazel_typescript//:index.bzl", "ts_library")

ts_library(
    name = "e2e_lib",
    testonly = True,
    srcs = [":app.spec.ts"],
    deps = [
        "@npm//@types/jasmine",
        "@npm//jasmine",
        "@npm//protractor",
    ],
)

protractor_web_test_suite(
    name = "e2e",
    on_prepare = ":protractor.on-prepare.js",
    server = "//src:devserver",
    deps = [":e2e_lib"],
)

Other common rules

Terser

yarn add -D @bazel/terser terser
load("@npm_bazel_terser//:index.bzl", "terser_minified")

terser_minified(
    name = "app_bundle_min",
    src = ":app_bundle.es2015.js",
)

filegroup

filegroup(
    name = "test_env_scripts",
    srcs = [
        "..."
    ],
)

alias

alias(
    name = "tsconfig.json",
    actual = "//src:tsconfig.json",
    visibility = ["//visibility:public"],
)

genrule

genrule(
    name = "copy",
    srcs = ["//somepackage:sometarget"],
    outs = ["output.js"],
    cmd = "cp $< $@",
)

glob

sh_test(
    name = "mytest",
    srcs = ["mytest.sh"],
    data = glob(
        ["testdata/*.txt"],
        exclude = ["testdata/experimental.txt"],
    ),
)

npm_package

npm_package(
    name = "npm_package",
    srcs = [
        "CHANGELOG.md",
        "README.md",
        "package.json",
    ],
    visibility = ["//visibility:public"],
    deps = [
        ":LICENSE.wrapped",
        ":LICENSE_copy",
        "//packages/app",
    ],
)

Angular with Bazel

Use @angular/bazel

# For existing project
ng add @angular/bazel

# For new project
npm install -g @angular/bazel
ng new --collection=@angular/bazel

Use @angular-devkit/architect-cli

load("@npm//@angular-devkit/architect-cli:index.bzl", "architect")

architect(
    name = "build",
    args = [
        "frontend:build",
        "--outputPath=$(@D)",
    ],
    ...
)

Difference

tsc

@angular/compiler-cli 

tsc-wrapped

@angular-devkit/architect

angular bazel

architect

angular_ts_library

Full stack demo

Troubleshooting

  • Check bazel-out folder
  • Environment issues
    • bazel clean
    • bazel clean --expunge
    • bazel shutdown

Thank you

References

Made with Slides.com