Rusty Utilities

Building Powerful Command Line Applications in Rust

Slides available at slides.com/kevinknapp/rusty_utilities

'talk: {

  • Why Rust
  • Command Line Uitilities
  • Command Line Interfaces using clap

}

  • Rust Basics
  • CLI UX
  • Environmental Variables
  • Configuration Files

$ whoami

Kevin Knapp

 

Systems & Network Admin

 

Avid skydiver

 

.NETer->Pythonista->Gopher->Rustacean

//

kbknapp

//

thekbknapp

There's more than one way to skin a cat...

Command Line Utilities in Rust

In 30 seconds...

Major Components

  • Controller
  • Interface
  • Configuration
  • Application
  • [Utilities]
  • Output
    • Errors
    • Results

Controller

  • The single point where execution starts and ends

  • Typically does the following in serial:
    • Builds the Interface
    • Parses the Interface results into a Configuration
    • Hands off the configuration to the Application
    • Displays any terminal results and output
    • Exits execution
mod cli;

fn main() {
    if let Err(e) = cli::parse().map(Config::from)
                                .and_then(run) {
        e.display_and_exit();
    }
}

fn run(c: Config) -> Result<Error> {
    // do the real stuff

    Ok(())
}

Interface

  • The language between the user and program
  • Can validate input format
  • Will normalize user input into a valid result
  • Types of interfaces include
    • Curses based
    • Single run
    • Interactive​
    • Hybrids
  • More to follow...

Configuration

  • A normalized representation of all program settings, input, and functionality
  • Often dictates how a program will function
  • Is typically built from the results of the interface parsing
  • Consider owned vs borrowed values
  • Consider mutability
struct Config {
    files: Vec<PathBuf>,
    pretty: bool,
    verbose: VerbLevel
}

Application

  • Does the things
  • Uses Result<T> idioms

As far as this talk is concerned...

Output

  • Errors or Results
  • Displayed inline or propagated up

Keep the user in mind!

Results

  • Typically written to stdout
    • Allows redirecting and piping
  • Prefer write! and writeln! over print! and println!
    • println! macros can panic! but writeln! macros return Result<T>s
    • writeln! macros allow abstracting over any io::Write or fmt::Write objects
  • Consider customisable output if applicable
    • Changing formats
    • Machine readable (JSON, YAML, CSV, etc.)

Errors

  • Arguably the most important information to return
  • Consider error-chain
  • Use distinct mod
  • Consider an enum
  • Impl From<E> where E is an expected external error (io, etc.)
  • Impl std::error::Error for MyError
  • Is the error terminal?
// From globset

#[derive(Clone, Debug, Eq, PartialEq)]
pub enum Error {
    UnclosedClass,
    InvalidRange(char, char),
    Regex(String),
    // snip..
}

Utilities

  • Grouped sets of stand alone "helpers"
  • Should be separate mod
  • If big enough, or autonomous, consider separate crate

Command Line Interfaces in Rust

clap

Command Line Argument Parser

kbknapp/clap-rs

clap.rs

It is a simple-to-use, efficient, and full-featured library for parsing command line arguments and subcommands when writing console/terminal applications.

//

//

Think of it as encoding a type system into arguments...

The Basics

Getting Started

  • Add clap to Cargo.toml dependencies array

 

 

 

  • Add clap to the top level project (src/main.rs)
[dependencies]
clap = "2.20.4"
// src/main.rs

extern crate clap;

Key Players

Builder Pattern

  • Allows a large number of [optional] inputs/configuration
  • Negates needing many different constructors
  • Typically finalizes construction via a "terminal method"
  • Two types of Builders
    • Consuming (terminal method takes self)
      • clap is a consuming builder*
    • Non-Consuming (terminal method takes &self or &mut self)

App

  • Contains all the valid Args, subcommands, etc.
  • Recursive tree-like structure
  • Can contain metadata
  • Generates a help message and two args automatically
    • -h/--help
    • -V/--version
    • Can be overriden
  • Terminal method: get_matches() 
extern crate clap;

use clap::App;

fn main() {
    App::new("prog")
        .author("Kevin K.")
        .version("1.1")
        .about("Does awesome things!")
        .get_matches();
}

(Run Online)

Arg

  • Represents a valid argument
  • Various settings determine final "type"
  • Three "types" of args
    • Flags
    • Options
    • Postional/Free
// A flag that can be used -F
Arg::with_name("flag")
    .help("I describe a flag")
    .short("F")

// An option that can be used --option <val>
Arg::with_name("opt")
    .help("Something about this option")
    .long("option")
    .takes_value(true)

// A positional argument that can be used <val>
Arg::with_name("arg")
    .help("I'm free!")

(Run Online)

Adding Args

  • Can be added one at a time, or in clusters
App::new("prog")
    .arg(Arg::with_name("flag")
        .help("I describe a flag")
        .short("F")
    )
    .args(&[
        Arg::with_name("opt")
            .help("Something about this option")
            .long("option")
            .takes_value(true),
        Arg::with_name("arg")
            .help("I'm free!")
    ])
    .get_matches();

(Run Online)

I can define args...

ArgMatches

  • The parsed structure of all valid args for that command
  • A recursive tree-like structure
  • Values, number of occurrences, or pressence are accessed by name
    • Values are stored in order
let m = App::new("prog")
    // ...
    .get_matches();

// Testing presence
println!("Was -F used?...{:?}", m.is_present("flag"));

// Getting a value
if let Some(val) = m.value_of("arg") {
    println!("<arg> has the value {}", val);
}

// Checking occurrences
println!("--option <val> was used {} times", m.occurrences_of("opt"));

(Run Online)

A little more detail...

Common Arg settings We've Seen So Far

  • short
    • Accessed via -X
    • Stackable, -xFZd == -x -F -Z -d
  • long
    • Accessed via --some
  • help
    • Displayed with help message
  • index
    • Starts at 1 not 0
    • Inferred if not specified manually
  • takes_value
    • Stackable with shorts: -xFsval == -x -F -sval
    • Values associated via
      • --some val
      • --some=val
      • -sval
Arg::with_name("opt")
    // ..snip
    .required(true)
Arg::with_name("arg")
    // ..snip
    .default_value("lol")
Arg::with_name("flag")
    // ..snip
    .multiple(true)

}

(Run Online)

Some new basics

* When combined with options, must consider multiple values vs multiple occurrences

SubCommands

  • Sub Apps. Period.
  • Metadata is independent of parent
  • Added the same as Args
App::new("git")
    .version("1.0")
    .about("does git things")
    .arg(Arg::with_name("work-tree")
        .long("work-tree")
        .takes_value(true))
    .subcommand(SubCommand::with_name("clone")
        .version("2.0")
        .about("clones things")
        .arg(Arg::with_name("repo")
            .help("The repo URL to clone")))
    .get_matches();

(Run Online)

SubCommand Trees

             Top Level App (git)                         TOP
                             |
      -----------------------------------------
     /             |                \          \
  clone          push              add       commit      LEVEL 1
    |           /    \            /    \       |
   url      origin   remote    ref    name   message     LEVEL 2
            /                  /\
         path            remote  local                   LEVEL 3

$ git clone url
$ git push origin path
$ git add ref local
$ git commit message

Valid Invocations

ArgMatches and SubCommands

  • Recursive tree-like structure
  • Requesting the ArgMatches from SubCommand gives back an independant ArgMatches

(Run Online)

let m = App::new("git")
    // ...
    .get_matches();

match m.subcommand() {
    ("clone", Some(clone_matches)) =>{
        println!("Cloning {}", clone_matches.value_of("repo").unwrap());
    },
    ("push", Some(push_matches)) =>{
        match push_matches.subcommand() {
            ("remote", Some(remote_matches)) =>{
                println!("Pushing to {}", remote_matches.value_of("repo").unwrap());
            },
            ("local", Some(_)) =>{
                println!("'git push local' was used");
            },
            _            => unreachable!(),
        }
    },
    ("add", Some(add_matches)) =>{
        println!("Adding {}", add_matches.values_of("stuff")
                                         .unwrap()
                                         .collect::<Vec<_>>()
                                         .join(", "));
    },
    ("", None)   => println!("No subcommand was used"), 
    _            => unreachable!(), 
}

Intermediate

...Customization...

Argument Relationships

Customizing a Single Value

  • Or allow empty values which allows --option= or --option ""
  • We can also give values names for help messages and errors
Arg::with_name("mode")
    .takes_value(true)
    .possible_value("fast")
    .possible_values(&[
        "slow",
        "medium"
    ])
Arg::with_name("config")
    .long("config")
    .value_name("FILE")

// produces --config <FILE>

(Run Online)

(Run Online)

Customizing Multiple Values

App::new("do")
    .arg(Arg::with_name("cmds")
        .multiple(true)
        .allow_hyphen_values(true)
        .value_terminator(";"))
    .arg(Arg::with_name("location"))

// run with: $ do find -type f -name special ; /home/clap

(Run Online)

Validators

  • The other feature that lets clap validate FOR YOU
  • Perform a function on a value to ensure it's validity
fn is_png(val: String) -> Result<(), String> {
    if val.ends_with(".png") {
        Ok(())
    } else {
        Err(String::from("the file format must be png."))
    }
}

App::new("myapp")
    .arg(Arg::with_name("input")
        .help("the input file to use")
        .validator(is_png))

(Run Online)

Let's Try It...

Simplified IPv4 address

  • Consists of four octets with numbers 0-255
  • Delimited by a dot "."
extern crate clap;  
use clap::{App, Arg};  
fn main() {  
    let m = App::new("net")
                 .arg(Arg::with_name("ip")
                    .short("i")
                    .long("ip-addr")
                    .value_name("X.X.X.X")
                    .help("An IP Address")
                    .required(true)
                    .number_of_values(4)
                    .validator(valid_octet)
                    .value_delimiter(".")
                    .require_delimiter(true))
                 .get_matches();

    let ip = m.values_of("ip").unwrap().collect::<Vec<_>>().join(".");
    println!("{} is probably a valid IP", ip);
}

fn valid_octet(o: String) -> Result<(), String> {  
    if o.parse::<u8>().is_err() {
        return Err(format!("'{}' must be a number between 0 and 255", o));
    }
    Ok(())
}

(Run Online)

Customizing an App with AppSettings

  • Can change behavior or just aesthetics
app.setting(AppSettings::GlobalVersion);

// or

app.settings(&[
    AppSettings::Hidden,
    AppSettings::ArgRequiredElseHelp
]);
  • Can also be applied globally which applies to all child SubCommands as well

(Run Online)

Notable AppSettings

  • AllowLeadingHyphen

  • DeriveDisplayOrder

  • Hidden

  • SubcommandsNegateReqs

  • {Subcommand, Arg}RequiredElseHelp

  • SubcommandRequired

  • UnifiedHelpMessage

  • *VersionlessSubcommands

  • *GlobalVersion

*Automatically applied globally

Customizing the Help with Templates

  • Help text can be supplemented with text before/after the message

  • OR broken into specific chunks

app.template(
"{bin} - v{version}
DESCRIPTION:
    {about}

USAGE:
    {usage}

AUTHOR:
    {author}

ARGS:
    {positionals}

OPTIONS:
    {flags}
    {options}

COMMANDS:
    {subcommands}
")

(Run Online)

Advanced Topics

Getting the most out of it

Conditionals

Aliases

  • Allows one to link SubCommands or Args
  • These can be shown in the help text (visible_alias) or hidden from the help text
  • When used, it's no different than original arg/subcommand being called
  • Great for
    • CLI migrations
    • UX (was it --completion or --completions??)

(Run Online)

App::new("alias")
    .arg(Arg::with_name("flag")
        .long("flag")
        .alias("flags")
        .visible_alias("flutter"))
    .subcommand(SubCommand::with_name("doc")
        .alias("docs")
        .visible_alias("documentation"))

Shell Completions

  • Supports Bash, Zsh, Fish, and PowerShell
  • Can be generated at compile time or runtime

Compile Time Completions

 

// src/cli.rs
use clap::{App, Arg, SubCommand};

pub fn build_cli() -> App<'static, 'static> {
    App::new("compl")
    // ..snip
}
# Cargo.toml
build = "build.rs"

[build-dependencies]
clap = "2.9"
// build.rs
extern crate clap;
use clap::Shell;
include!("src/cli.rs");

fn main() {
    let mut app = build_cli();
    for shell in &Shell::variants() {
        app.gen_completions("myapp", shell.parse().unwrap(), env!("OUT_DIR"));
    }
}

Runtime Completions

 

extern crate clap;

use std::io;

use clap::{App, Arg, SubCommand, Shell};

fn build_cli() -> App<'static, 'static> {
    App::new("My Super Program")
	// ..snip
    .subcommand(SubCommand::with_name("completions")
          .arg(Arg::with_name("shell")
            .possible_values(&Shell::variants())
            .required(true)))
}

fn main() {
    let m = build_cli().get_matches();

    if let Some(compl_m) = m.subcommand_matches("completions") {
        let mut app = build_cli();
        let shell = compl_m.value_of("shell").unwrap().parse().unwrap();
        app.gen_completions_to("demo", shell, &mut io::stdout());
    }
}

(Run Online)

Key Topics Not Covered

Still to come...

Current Tracking Issues

  • Manpage Generation - #552
  • Automatic Negation Flags - #815
  • Deserialization - #146 / #817 / #835 (PR)
  • enum variants as Arg keys - #459
  • Values from config files - #748
  • Values from ENV vars - #712
  • Rust code for custom shell completions - #568
  • Add color support for Windows - #836

Credits

Get Involved!

Contact / Follow Up

Made with Slides.com