Bazel Introduction
@JiaLiPassion at ngTaiwan 2019
Who am I
- Name: Jia Li
- Company: @ThisDot
- Zone.js: Code Owner
- Angular: Collaborator
- @JiaLiPassion
Bazel
- Bazel introduction
- Bazel in TypeScript project
- Bazel in Angular
What is Bazel
- A Build Tool
- Incremental
- For any languages/Frameworks
- Scalable
Bazel is a replacement of Jenkins or Webpack?
- CI: Jenkins/CircleCI/...
- tsc/rollup/webpack/sass/...
- make/grunt/gulp/...
AlexEagle at ngconf 2019
Bazel is Hub
AlexEagle at ngconf 2019
Why not just continue to use gulp
The goal for Build tool
- Essential:
- Correct - don't need to worry about environment
- Fast
- Incremental
- Parallelism
- Predictable - same input -> same output
- Reusable -> build rule/task can be composed and reused
- Nice to have:
- Universal -> support multiple languages/framework
Correctness
- Bazel use sandbox to isolate the environment
- Sandbox support Linux and Mac (Some features not supported)
Universal
- Bazel doesn't rely on any specified framework or languages
- Bazel is a perfect build tool for full stack development inside mono repo.
Gulp is imperative
gulp.task('build', () => {
return gulp.src(['src/*.ts'])
.pipe(tsc(...))
.pipe(gulp.dest('../build/'));
})
gulp.task('test', ['build'], () => {
...
});
- gulp will run build task
- gulp will run test task
Bazel is declarative
Sample
// date.ts
export function formatDate(d: Date) {
return `${d.getFullYear()}-${d.getMonth()}-${d.getDate()}!!`;
}
// user.ts
import { formatDate } from './date';
/** Data object representing a User. */
export class User {
constructor(readonly name: string, readonly birthday: Date) {}
toString() {
return `${this.name}, born on ${formatDate(this.birthday)}`;
}
}
// birthday_card.ts
import { User } from './user';
/** Prints a birthday greeting for the given user to the console. */
export function printBirthdayGreeting(user: User) {
console.log(`Happy birthday ${user.name}, ${user.birthday} is your day!`);
}
Dependencies
date.ts
user.ts
birthday_card.ts
Gulp is imperative
date.ts task
user.ts task
birthday_card.ts
task
Bazel is declarative
date.ts target
user.ts target
birthday_card.ts
target
Bazel target
# date typescript build target
ts_library(
name = "date",
srcs = ["date.ts"],
)
date.ts target
input
date.ts
ts_library rule
date
output
1. date.d.ts
2. date.js
Bazel target
# date typescript build target
ts_library(
name = "date",
srcs = ["date.ts"],
)
# user typescript build target
ts_library(
name = "user",
srcs = ["user.ts"],
deps = [":date"]
)
date target
user rule
input
date.ts
ts_library rule
date
output
1. date.d.ts
2. date.js
input
user.ts
input
date.d.ts
ts_library rule
user
output
1. user.d.ts
2. user.js
Demo
Predictive
result = f(input)
Bazel dependency graph
Bazel
-
Incremental - Analyze and only run really impacted targets
-
Deterministic - We can cache build results because rules are pure function
-
Hermetic - remote execution, parallelization
-
Composable - Bazel plugins are like Unix pipes
-
Universal - Builds Android, iOS, web, backends, cloud services, and more (in theory everything)
-
Industrial grade - All google big mono repo projects are using Babel
Setup Bazel
- npm install
- Prebuilt binary download
npm install globally
$ npm i -g @bazel/bazel
Download preload binary
Install Bazel from https://bazel.build.
Windows needs to install MSYS2, WSL are not supported yet.
Concept and Terminology
- Workspace
- Package
- Label
- Rule
- Build File
Structure
WORKSPACE
Package
Target
Workspace
- Directory in the file system contains
- source files of your project
- symbolic links that contains your build output
- workspace definition is in a file named WORKSPACE at the root of the directory.
WORKSPACE
workspace(
name = "com_taiwan_2019",
managed_directories = {"@npm": ["node_modules"]},
)
load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
http_archive(
name = "build_bazel_rules_nodejs",
sha256 = "0942d188f4d0de6ddb743b9f6642a26ce1ad89f09c0035a9a5ca5ba9615c96aa",
urls = ["https://github.com/bazelbuild/rules_nodejs/releases/download/0.38.1/rules_nodejs-0.38.1.tar.gz"],
)
1. workspace name should be unique globally. Use something like reverse dns name such as (com_ngtaiwan_2019)
2. in WORKSPACE file, we should do
- install env such as yarn/npm/bazel
- setup toolchain such typescript
Packages
- The primary unit of code organization(something like module) in a repository
- Collection of related files and a specification of the dependencies among them
- Directory containing a file named BUILD or BUILD.bazel, residing beneath the top-level directory in the workspace
- A package includes all files in its directory, plus all subdirectories beneath it, except those which themselves contain a BUILD file
Packages
Gulp structure
gulp.task('build-animations', () => {});
gulp.task('build-core', () => {});
gulp.task('build-core-schematics', () => {});
...
gulp.task('test-animations', () => {});
1. gulp file doesn't have 1:1 relationship to module directory.
2. gulp can reference files from any directories in gulp task.
Bazel Packages
# WORKSPACE file is at the root of directory
├── WORKSPACE
├── BUILD.bazel
├── ...
├── animations
│ ├── BUILD.bazel
│ ├── ...
│ ├── browser
│ │ └── BUILD.bazel
| | ...
│ ├── test
│ │ ├── BUILD.bazel
│ │ └── ...
# root package
# animations package
# animations/browser package
# animations/test package
BUILD file
Starlark language
def fizz_buzz(n):
"""Print Fizz Buzz numbers"""
for i in range(1, n + 1):
s = ""
if i % 3 == 0:
s += "Fizz"
if i % 5 == 0:
s += "Buzz"
print(s if s else i)
fizz_buzz(20)
1. Subset of Python
2. class, import, while, yield, lambda, is, raise, etc not supported
3. Recursion not allowed
4. Most of python builtin methods not supported
BUILD file
- contains build targets
- build targets can be
- files
- rules
- target can depend on other target
- circular dependency are not allowed
- two target generate same output will cause problem
BUILD file
package(default_visibility = ["//visibility:private"])
load("@npm_bazel_typescript//:index.bzl", "ts_library")
ts_library(
name = "lib",
srcs = [":lib.ts"],
visibility = ["//visibility:public"],
)
Label: name of the target
@com_ngtaiwan_2019//lib:build
Workspace name
package
target
shorthand without WORKSPACE
//lib:build
package
target
shorthand without target
//lib
package
//lib:lib
package
target
package visibility
package(default_visibility = ["//visibility:private"])
load("@npm_bazel_typescript//:index.bzl", "ts_library")
ts_library(
name = "lib",
srcs = [":lib.ts"],
visibility = ["//visibility:public"],
)
- private
- public
- //some_package:__subpackages__
import
package(default_visibility = ["//visibility:private"])
load("@npm_bazel_typescript//:index.bzl", "ts_library")
ts_library(
name = "lib",
srcs = [":lib.ts"],
visibility = ["//visibility:public"],
)
- import rule
- import macro
target
package(default_visibility = ["//visibility:private"])
load("@npm_bazel_typescript//:index.bzl", "ts_library")
ts_library(
name = "lib",
srcs = [":lib.ts"],
visibility = ["//visibility:public"],
)
Buildifier
yarn add -D @bazel/buildifier
"scripts": {
"bazel:format": "find . -type f \\( -name \"*.bzl\" -or -name WORKSPACE -or -name BUILD -or -name BUILD.bazel \\) ! -path \"*/node_modules/*\" | xargs buildifier -v --warnings=attr-cfg,attr-license,attr-non-empty,attr-output-default,attr-single-file,constant-glob,ctx-actions,ctx-args,depset-iteration,depset-union,dict-concatenation,duplicated-name,filetype,git-repository,http-archive,integer-division,load,load-on-top,native-build,native-package,out-of-order-load,output-group,package-name,package-on-top,positional-args,redefined-variable,repository-name,same-origin-load,string-iteration,unsorted-dict-items,unused-variable",
"bazel:lint": "yarn bazel:format --lint=warn",
"bazel:lint-fix": "yarn bazel:format --lint=fix"
}
package.json
Bazel Query
Bazel Query
- Bazel can statically analyze BUILD.bazel files
- Using Bazel Query to understand the project build structure (can output graph)
- Bazel query can also return results for further operations.
Sample Queries
# All the dependencies of //src/app
bazel query 'deps(//src/app)'
# All the targets in //src
bazel query '//src:*'
# Path between any of the targets in src/app/... and any in //src/lib
bazel query 'somepath(foo/..., //bar/baz:all)'
# Reverse dependencies of //src/lib within the transitive closure
# of the universe set //src/app
bazel query 'rdeps(//src/app, //src/lib:date)'
Query targets
# All the targets in //src/lib
bazel query '//src/lib:*'
# All rules in //src
bazel query '//src/lib:all'
# All targets under //src
bazel query '//src/...'
- :* will match all targets (files and rules)
- :all will only match all rules
- ... will include all sub packages.
Query dependencies
# query dependencies of //src/app
bazel query 'deps(//src/app)'
# why //src/app depends on //src/lib
bazel query 'somepath(//src/app, //src/lib)'
bazel query 'allpaths(//src/app, //src/lib)'
# query who depends on //src/lib
bazel query 'rdeps(..., //src/lib)'
- somepath will return any possible path (only one)
- allpaths will return all paths.
- reverse dependency query may very expensive.
Tag
ts_library(
name = "lib",
srcs = [":index.ts"],
tags = ["build-target"],
visibility = ["//visibility:public"],
deps = [
":date",
":user",
],
)
Query by tags
# query all targets with tags "build-target" under //src
bazel query --output=label 'attr("tags", "\[.*build-target.*\]", //src/...)'
complex query
# query all targets with tags "build-target" under //src and label must
# ends with lib
bazel query --output=label 'attr("tags", "\[.*build-target.*\]", //src/...)
intersect kind(".*lib", //src/...)'
- intersect
- except
- union
- ...
Bazel query Visualize
# Show graphical representation of the build graph
bazel query --output=graph ... | dot -Tpng > graph.png
need to install graphviz
Target
Build Target
- Files/Rules Target
- Files such as tsconfig.json
- Rules such as ts_library()
- Target input/output are known at build time
- Target will only be built when input changed.
Rule Target Naming
- *_binary
- executable programs in a given language (nodejs_binary)
- *_test
- special _binary rule for testing
- *_library
- compiled module for given language (ts_library)
Rule Target
ts_library(
name = "lib",
srcs = [":index.ts"],
tags = ["build-target"],
visibility = ["//visibility:public"],
deps = [
":date",
":user",
],
)
Common attributes
- name - unique name within this package
- srcs - inputs of the target, typically files
- deps - compile-time dependencies
- data - runtime dependencies
- testonly - target which should be executed only when running bazel test
- visibility - specifies who can make a dependency on the given target
Data: runtime dep
http_server(
name = "prodserver",
data = [
"index.html",
":bundle",
"styles.css",
],
)
- data should only be used at runtime
- data will not be analyzed at build time
bazel build
bazel build
bazel build //src/app
bazel build output
bazel info
- bazel-bin: build files
- output_path: all output artifacts during the process
Bundle with rollup
yarn add -D @bazel/rollup rollup
load("@build_bazel_rules_nodejs//:defs.bzl", "rollup_bundle")
rollup_bundle(
name = "app_bundle",
entry_point = "//src/app:birthday_card.ts",
deps = ["//src/app"],
)
bazel build //src:app_bundle
Request different formats rollup
bazel build //src:app_bundle.es2015.js
bazel build //src:app_bundle.umd.js
bazel build //src:app_bundle.min.umd.js
Dev Server
watch mode
yarn add -D @bazel/ibazel
ts_devserver
ts_devserver(
name = "devserver",
index_html = "index.html",
port = 4200,
deps = [
":app_bundle.es2015.js",
],
)
Jasmine test
jasmine_node_test
yarn add -D @bazel/jasmine jasmine @types/jasmine
jasmine_node_test rule
load("@npm_bazel_jasmine//:index.bzl", "jasmine_node_test")
load("@npm_bazel_typescript//:index.bzl", "ts_library")
ts_library(
name = "test_lib",
testonly = True,
srcs = [":date.spec.ts"],
deps = [
"//src/lib:date",
"@npm//@types/jasmine",
],
)
jasmine_node_test(
name = "test",
deps = [":test_lib"],
)
bazel test
bazel test //src/...
bazel test //src/lib/test
Debug jasmine test
test:debug --test_arg=--node_options=--inspect-brk --test_output=streamed --test_strategy=exclusive --test_timeout=9999 --nocache_test_results
.bazelrc
bazel test //src/lib/test --config=debug
// open chrome://inspect
Karma test
Karma setup
yarn add -D @bazel/karma karma
# Load karma dependencies
load("@npm_bazel_karma//:package.bzl",
"npm_bazel_karma_dependencies")
npm_bazel_karma_dependencies()
# Set up web testing, choose browsers we can test on
load("@io_bazel_rules_webtesting//web:repositories.bzl",
"browser_repositories", "web_test_repositories")
web_test_repositories()
browser_repositories(
chromium = True,
)
Karma rule
load("@npm_bazel_karma//:index.bzl", "karma_web_test_suite")
ts_library(
name = "test_lib",
testonly = True,
srcs = [":date.spec.ts"],
deps = [
"//src/lib:date",
"@npm//@types/jasmine",
],
)
karma_web_test_suite(
name = "test_web",
deps = [":test_lib"],
)
run/debug karma
bazel test //src/lib/test:test_web
bazel run //src/lib/test:test_web
Protractor
Protractor setup
yarn add -D protractor @bazel/protractor
Protractor rule
load("@npm_bazel_protractor//:index.bzl", "protractor_web_test_suite")
load("@npm_bazel_typescript//:index.bzl", "ts_library")
ts_library(
name = "e2e_lib",
testonly = True,
srcs = [":app.spec.ts"],
deps = [
"@npm//@types/jasmine",
"@npm//jasmine",
"@npm//protractor",
],
)
protractor_web_test_suite(
name = "e2e",
on_prepare = ":protractor.on-prepare.js",
server = "//src:devserver",
deps = [":e2e_lib"],
)
Other common rules
Terser
yarn add -D @bazel/terser terser
load("@npm_bazel_terser//:index.bzl", "terser_minified")
terser_minified(
name = "app_bundle_min",
src = ":app_bundle.es2015.js",
)
filegroup
filegroup(
name = "test_env_scripts",
srcs = [
"..."
],
)
alias
alias(
name = "tsconfig.json",
actual = "//src:tsconfig.json",
visibility = ["//visibility:public"],
)
genrule
genrule(
name = "copy",
srcs = ["//somepackage:sometarget"],
outs = ["output.js"],
cmd = "cp $< $@",
)
glob
sh_test(
name = "mytest",
srcs = ["mytest.sh"],
data = glob(
["testdata/*.txt"],
exclude = ["testdata/experimental.txt"],
),
)
npm_package
npm_package(
name = "npm_package",
srcs = [
"CHANGELOG.md",
"README.md",
"package.json",
],
visibility = ["//visibility:public"],
deps = [
":LICENSE.wrapped",
":LICENSE_copy",
"//packages/app",
],
)
Angular with Bazel
Use @angular/bazel
# For existing project
ng add @angular/bazel
# For new project
npm install -g @angular/bazel
ng new --collection=@angular/bazel
What changed
"architect": {
"build": {
"builder": "@angular/bazel:build",
...
}
}
IDE Integration
vscode-bazel
Troubleshooting
- Check bazel-out folder
- Environment issues
- bazel clean
- bazel clean --expunge
- bazel shutdown
Thank you
References
- bazel presentation by Martin Probst at Angular connect 2018
- bazel introduction by Alex Eagle at ng-conf 2019
- bazel training by Minko
- https://angular.io/guide/bazel
- https://github.com/JiaLiPassion/ngtaiwan2019-bazel
- @JiaLiPassion
Bazel ngtaiwan 2019
By jiali
Bazel ngtaiwan 2019
- 1,166