Zephyr Manifest Files

Project Sanity in the

Ever-changing Zephyr World

Zephyr Manifest Files

Project Sanity in the

Ever-changing Zephyr World

tl;dr:

Manifest files deliver the best revision control for your project

Lets talk about

  • What Zephyr's "manifest" files are
  • How they work
  • Why we should use them in project repos
  • How to set up a manifest-based Zephyr application
  • Next-step features you should know and use
  • Things that exist outside the scope of this talk

@szczys@chaos.social on Mastodon

@szczys@bsky.social on Bluesky

  • Developer Relations Engineer at Golioth
  • 15 years of firmware experience
  • Previously: Editor in Chief of Hackaday

Mike Szczys

What is Golioth?

  • Golioth is an IoT cloud company.

  • We make it easy for hardware engineers to connect their sensors and devices to the web without needing to be a cloud expert.

  • We use Zephyr to support the widest range of hardware possible. Manifest files are one big piece of that puzzle.

What is a manifest file?

What is a manifest file?

  • Manages Project heiarchy

  • Repositories:

    • URL of each repo

    • Repo branch/tag/commit

    • Local path for checkout

  • Committed to a codebase

Advanced Manifest Features:

  • Inheritance

  • Allow/Block lists

  • Grouping

  • Repo branch/tag/commit

  • West support for validation

Applications 💖 Manifests

  • Zephyr tree uses Manifests to manage versions of modules and libraries

  • Projects can (and should) follow the same paradigm

If you are not using

manifests in your projects...

You're Doing Zephyr Wrong!!

Manifest Examples

Main Zephyr Manifest:

356 Lines

./west.yml

Basic Structure

Project Based on Default

Project with Explicit Path

Inheritance, allowlist, command

Inheritance

Entire nRF Connect SDK Manifest (if on allowlist)

Use West Init

(not Git Clone)

west init <arguments>

  • Get manifest from repository

  • Clones the top-level repository

  • Creates .west/config file with path to top-level manifest and Zephyr location

west update

  • Parse manifest file

  • Clone repositories

  • Reset repos to specified revision

  • Recurse

Running west init

Running west init

West Init Gotcha

Once you've run west init (even if it fails) you cannot run that command again without removing .west/config

Running west update

West Update Gotcha

If you checkout a new branch/tag/commit your manifest file might change with that checkout!

 

Run west update religiously.

 

Be compusive about! (Seriously!)

Revision Control for

Your Zephyr Applications

Revision Control for

Your Zephyr Applications

Plan your applications so they will always have the build environment present when the code was committed.

Do Not: Fork the Zephyr tree for your project repo

Not Ideal: App repo in Zephyr samples folder

         It will work, but your app has

no way to track what version of Zephyr should be used for the build.

Not Ideal: App repo with external Zephyr tree

         Again, this will work. But what version of Zephyr is being used for the build?

project repo

Zephyr location

  • Your project is in the app folder*
  • Zephyr tree/modules in deps folder*
  • Everything is controlled by manifest files
  • Your project repo consists only of the contents of app

The Right Wayâ„¢

*names for app/dep folders can be anything

Pros:

  • Zephyr and all modules locked to a known version
  • Your repo only tracks your files
  • Zephyr tree/modules in deps folder
  • Build environment is easily replicated*

The Right Wayâ„¢

Cons:

  • Every Zephyr application stores its own local copy of Zephyr and its modules (~2.2 GB and larger)
  • VS Code extensions are playing catch-up

*But not the Zephyr SDK and Python

The Right Wayâ„¢

Thank you to

Asgeir Stavik Hustad

Setting Up Your Zephyr Application

Setting Up Your Zephyr Application

  • Create west.yml with app directory

  • Add Zephyr in the deps

  • Add Modules

  • Limit modules with allowlist

Simple beginnings

Our repo will be cloned to an app subfolder of the folder containing .west/config

Add Zephyr

Adds Zephyr tree to dep/zephyr

(but no modules)

We Need Modules

This won't build:

Import from Zephyr

This will download ALL

Zephyr Modules (yuck!)

        Replaces

path: deps/zephyr

path-prefix was added by west manifest v0.8.0

Removed

Limit the Zephyr import

Limit to only the modules needed by this project

west update with allow-list

Our Zephyr Application Tree

Notice .git, this is your application repo

We don't need Git to track this, the manifest file in our application repo does this!

Customize Your Applications

Add External Library

Problem: are there any other modules needed by the library we're adding?

Solution: Use the manifest from your library to figure this out

Libraries: Matching Version

Do these versions work together?

Yes, because we manually matched them.

But... let's look at the manifest(s) from this library

Libraries: Matching Version

This library manifest includes an entry for a tested-working version of Zephyr

This is the library's manifest file:

We can use inheritance to always pull the correct Zephyr version!

Libraries: Matching Version

Original Application Manifest

Improved Application Manifest

Add Your Own Helper Libraries

Custom board definitions

Out-of-tree driver library

Whew, that was a lot!

  • Most things are simple (don't be overwhelmed)

  • If you get stuck, look for how different Zephyr modules use manifest tricks

  • Often, things can be done in more than one way

Other Manifest Features You Should Know About

(But we have no time to discuss them)

Resolve

west manifest --resolve

  • Produce a single manifest from source and all imported manifests
  • Great for verifying your modules and local paths

Freeze

west manifest --freeze

  • Produce a single manifest from source and all imported manifests
  • All revisions are commit hashes (branches and tags will be converted)

Groups

  • Associate different modules into groups
  • Filter your manifests based on groups
  • This is handy during import... like blocklist but for multiple modules

West Extensions

west <command> <param>

  • West extensions allow you to add your own custom commands to west
  • These custom commands can be imported from modules

User Data

  • Passes data into the project available using west
  • Golioth uses this with Python-based west extension (custom command) to apply patches for Twister-based testing

Summary:

  • Manifests are an important part of revision control

  • Proper use stabilizes the build environment

  • Useful for out-of-tree boards/drivers/libraries

  • Application: https://github.com/golioth/zephyr-training
  • Library: https://github.com/golioth/golioth-zephyr-sdk

Read Zephyr Docs:

  • https://golioth.io

Manifest Examples:

  • https://docs.zephyrproject.org/latest/develop/west/manifest.html

Try Golioth:

My name is Mike Szczys, thank you for joining me!