Buildng

CLIs for Fun and Profit with Python

Aug 3, 2016

Who: Lucas Roesler

What: Director of Engineering

Where: Eventboard.io

Why: CLIs are fun!

Today we will see how to build command-line interfaces in python using

 

Honorable Mention

Awesome UI

Real Power

Why I wrote octoeb

Why

  • We use git-flow for branching
  • We use GitHub Releases for releases
  • We have had fat finger incidents
  • We feel branching ramp up is faster with octoeb

What

  • Create new branches, start new releases, start pull requests, generate a changelog, and print the current version numbers

Baby Steps

Say hello (in German)

# hallo.py
if __name__ == '__main__':
    print('Hallo Welt!')
(.env) ➜ python hallo.py
Hallo Welt!
(.env) ➜ python hallo.py Hanz Franz
Hallo Welt!

What about arguments?

How to say hello (part 2)

(.env) ➜  python hallo2.py Hanz Franz
Hallo Hanz!
Hallo Franz!

(.env) ➜  python hallo2.py Hanz Franz --bye
Tschüss Hanz!
Tschüss Franz!
# hallo2.py
import argparse

parser = argparse.ArgumentParser(description='Say Hello to someone')
parser.add_argument(
    'names',
    metavar='NAME', type=str, nargs='+',
    help='a person to speak to',
)
parser.add_argument('--bye', action='store_true', default=False)

if __name__ == '__main__':
    args = parser.parse_args()

    if args.bye:
        msg = 'Tschüss {}!'
    else:
        msg = 'Hallo {}!'

    for name in args.names:
        print(msg.format(name.capitalize()))

Was that really a CLI?

Was that really a CLI?

#! /usr/bin/env python
# -*- encoding: utf-8 -*-
# hallo3.py
import argparse

parser = argparse.ArgumentParser(description='Say Hello to someone')
parser.add_argument(
    'names',
    metavar='NAME', type=str, nargs='+',
    help='a person to speak to',
)
parser.add_argument('--bye', action='store_true', default=False)

if __name__ == '__main__':
    args = parser.parse_args()

    if args.bye:
        msg = 'Tschüss {}!'
    else:
        msg = 'Hallo {}!'

    for name in args.names:
        print(msg.format(name.capitalize()))
(.env) ➜  ln -sf ~/Code/slcpy-cli/beispiel/hallo3.py /usr/local/bin/hallo
(.env) ➜  hallo hanz franz
Hallo Hanz!
Hallo Franz!

Moar, Can Haz Moar!

Subcommands (be like git)

➜  git -h
Unknown option: -h
usage: git [--version] [--help] [-C <path>] [-c name=value]
           [--exec-path[=<path>]] [--html-path] [--man-path] [--info-path]
           [-p | --paginate | --no-pager] [--no-replace-objects] [--bare]
           [--git-dir=<path>] [--work-tree=<path>] [--namespace=<name>]
           <command> [<args>]

➜  git <TAB><TAB>
add       -- add file contents to the index
bisect    -- find by binary search the change that introduced a bug
branch    -- list, create, or delete branches
checkout  -- checkout a branch or paths to the working tree
clone     -- clone a repository into a new directory
commit    -- record changes to the repository
diff      -- show changes between commits, commit and working tree, etc
fetch     -- download objects and refs from another repository
grep      -- print lines matching a pattern
init      -- create an empty Git repository or reinitialize an existing one
log       -- show commit logs
merge     -- join two or more development histories together
mv        -- move or rename a file, a directory, or a symlink
pull      -- fetch from and merge with another repository or a local branch
push      -- update remote refs along with associated objects
rebase    -- forward-port local commits to the updated upstream head
reset     -- reset current HEAD to the specified state
rm        -- remove files from the working tree and from the index
show      -- show various types of objects
status    -- show the working tree status
tag       -- create, list, delete or verify a tag object signed with GPG

➜  git add -h
usage: git add [<options>] [--] <pathspec>...

    -n, --dry-run         dry run
    -v, --verbose         be verbose

    -i, --interactive     interactive picking
    -p, --patch           select hunks interactively
    -e, --edit            edit current diff and apply
    -f, --force           allow adding otherwise ignored files
    -u, --update          update tracked files
    -N, --intent-to-add   record only the fact that the path will be added later
    -A, --all             add changes from all tracked and untracked files
    --ignore-removal      ignore paths removed in the working tree (same as --no-all)
    --refresh             don't add, only refresh the index
    --ignore-errors       just skip files which cannot be added because of errors
    --ignore-missing      check if - even missing - files are ignored in dry run

Be Like Git (subcommands)

#! /usr/bin/env python
# -*- encoding: utf-8 -*-
# sagen.py
import argparse


def hallo(args):
    for name in args.names:
        print('Hallo {}!'.format(name.capitalize()))


parser = argparse.ArgumentParser(description='Speak messages to the cli')
subparsers = parser.add_subparsers(
    title='subcommands',
)


parser_hallo = subparsers.add_parser(
    'hallo', help='Print a German greeting to the cli')
parser_hallo.add_argument(
    'names',
    metavar='NAME', type=str, nargs='+',
    help='a persons name',
)
parser_hallo.set_defaults(func=hallo)


def bye(args):
    for name in args.names:
        print('Tschüss {}!'.format(name.capitalize()))


parser_bye = subparsers.add_parser(
    'bye', help='Print a German parting to the cli')
parser_bye.add_argument(
    'names',
    metavar='NAME', type=str, nargs='+',
    help='a persons name',
)
parser_bye.set_defaults(func=bye)


if __name__ == '__main__':
    args = parser.parse_args()
    args.func(args)
(.env) ➜ python sagen.py -h
usage: sagen.py [-h] {hallo,bye} ...

Speak messages to the cli

optional arguments:
  -h, --help   show this help message and exit

subcommands:
  {hallo,bye}
    hallo      Print a German greeting to the cli
    bye        Print a German parting to the cli

(.env) ➜ python sagen.py hallo hanz
Hallo Hanz!

Just one Click

#! /usr/bin/env python
# -*- encoding: utf-8 -*-
# sagen_click.py
import click

CONTEXT_SETTINGS = dict(help_option_names=['-h', '--help'])


@click.group(context_settings=CONTEXT_SETTINGS)
@click.version_option('1.0.0')
def cli():
    """Print a German phrases."""
    pass


@cli.command()
@click.argument('names', nargs=-1)  # See http://click.pocoo.org/6/arguments/
def hallo(names):
    """Print a German greeting to the cli"""
    for name in names:
        click.echo('Hallo {}!'.format(name.capitalize()))


@cli.command()
@click.argument('names', nargs=-1)  # See http://click.pocoo.org/6/arguments/
def bye(names):
    """Print a German parting to the cli."""
    for name in names:
        click.echo('Tschüss {}!'.format(name.capitalize()))


if __name__ == '__main__':
    cli()
(.env) ➜  python sagen_click.py -h
Usage: sagen_click.py [OPTIONS] COMMAND [ARGS]...

  Print a German phrases.

Options:
  --version   Show the version and exit.
  -h, --help  Show this message and exit.

Commands:
  bye    Print a German parting to the cli.
  hallo  Print a German greeting to the cli

(.env) ➜  python sagen_click.py hallo -h
Usage: sagen_click.py hallo [OPTIONS] [NAMES]...

  Print a German greeting to the cli

Options:
  -h, --help  Show this message and exit.

(.env) ➜  python sagen_click.py hallo hanz franz
Hallo Hanz!
Hallo Franz!

Click v Argparse

#! /usr/bin/env python
# -*- encoding: utf-8 -*-
# sagen_click.py
import click

CONTEXT_SETTINGS = {'help_option_names': ['-h', '--help']}


@click.group(context_settings=CONTEXT_SETTINGS)
@click.version_option('1.0.0')
def cli():
    """Print a German phrases."""
    pass


@cli.command()
@click.argument('names', nargs=-1)  
def hallo(names):
    """Print a German greeting to the cli"""
    for name in names:
        click.echo('Hallo {}!'.format(name.capitalize()))


@cli.command()
@click.argument('names', nargs=-1)  
def bye(names):
    """Print a German parting to the cli."""
    for name in names:
        click.echo('Tschüss {}!'.format(name.capitalize()))


if __name__ == '__main__':
    cli()
#! /usr/bin/env python
# -*- encoding: utf-8 -*-
# sagen.py
import argparse


def hallo(args):
    for name in args.names:
        print('Hallo {}!'.format(name.capitalize()))


parser = argparse.ArgumentParser(
    description='Speak messages to the cli')
subparsers = parser.add_subparsers(
    title='subcommands',
)


parser_hallo = subparsers.add_parser(
    'hallo', help='Print a German greeting to the cli')
parser_hallo.add_argument(
    'names',
    metavar='NAME', type=str, nargs='+',
    help='a persons name',
)
parser_hallo.set_defaults(func=hallo)


def bye(args):
    for name in args.names:
        print('Tschüss {}!'.format(name.capitalize()))


parser_bye = subparsers.add_parser(
    'bye', help='Print a German parting to the cli')
parser_bye.add_argument(
    'names',
    metavar='NAME', type=str, nargs='+',
    help='a persons name',
)
parser_bye.set_defaults(func=bye)


if __name__ == '__main__':
    args = parser.parse_args()
    args.func(args)

Let's get real

octoeb

➜  octoeb -h
Usage: octoeb [OPTIONS] COMMAND [ARGS]...

  Eventboard releases script

Options:
  --log [DEBUG|INFO|WARNING|ERROR|CRITICAL]
                                  Set the log level
  --version                       Show the version and exit.
  -h, --help                      Show this message and exit.

Commands:
  changelog  Get changelog between base branch and head...
  jira       (DEV) Call JiraAPI mehtod directly
  method     (DEV) Call GitHubAPI directly
  qa         Publish pre-release on GitHub for QA.
  release    Publish release on GitHub
  review     Create PR to review your code
  start      Start new branch for a fix, feature, or a new...
  sync       Sync fork with mainline Checkout each core...
  update     Update local branch from the upstream base...
  versions   Get the current release and pre-release...

octoeb (recap)

Why

  • We use git-flow for branching
  • We use GitHub Releases for releases
  • We have had fat finger incidents
  • We feel branching ramp up is faster with octoeb

What

  • Create new branches, start new releases, start pull requests, generate a changelog, and print the current version numbers

octoeb (pieces)

# cli.py

def validate_version_arg(ctx, param, version):
    if version is None:
        raise click.BadParameter('Version number is required')

    if re.match(r'^(?:\.?\d+){4,5}$', version):
        return version

    raise click.BadParameter('Invalid versom format: {}'.format(version))


@cli.group()
@click.pass_obj
def start(apis):
    """Start new branch for a fix, feature, or a new release"""
    pass


@start.command('release')
@click.option(
    '-v', '--version',
    callback=validate_version_arg,
    help='Major version number of the release to start')
@click.pass_obj
def start_release(apis, version):
    """Start new version branch"""
    api = apis.get('mainline')
    try:
        major_version = extract_major_version(version)
        name = 'release-{}'.format(major_version)
        branch = api.create_release_branch(name)
    except GitHubAPI.DuplicateBranchError as e:
        git.fetch('mainline')
        git.checkout(name)
        branch = api.get_branch(name)
        logger.debug('Branch already started')
    except Exception as e:
        sys.exit(e.message)

    try:
        git.fetch('mainline')
        log = git.log(
            'mainline/master', 'mainline/{}'.format(name), merges=True)
        changelog = git.changelog(log)

        click.echo('Changelog:')
        click.echo(changelog)

        logger.info('Creating slack channel')
        channel_name = 'release_{}'.format(major_version.replace('.', '_'))
        slack_create_url = (
            'https://zyhpsjavp8.execute-api.us-west-2.amazonaws.com'
            '/slack_prod/slack/slash-commands/release-channel'
        )
        logger.info('Channel name: {}'.format(channel_name))

        try:
            resp = requests.post(
                slack_create_url,
                data={
                    'channel_id': channel_name,
                    'text': channel_name
                })
            logger.debug(resp)
        except Exception as e:
            sys.exit(e.message)
        finally:
            logger.info('Tagging new version for qa')
            qa(apis, version)
    except Exception as e:
        sys.exit(e.message)
    finally:
        click.echo('Branch: {} created'.format(name))
        click.echo(branch.get('url'))
        click.echo('\tgit fetch --all && git checkout {}'.format(name))

    sys.exit()

octoeb (pieces)

# cli.py

def set_logging(ctx, param, level):
    numeric_level = getattr(logging, level.upper(), None)
    if not isinstance(numeric_level, int):
        raise click.BadParameter('Invalid log level: {}'.format(level))

    logging.basicConfig(level=numeric_level)


@click.group(context_settings=CONTEXT_SETTINGS)
@click.option(
    '--log',
    default='ERROR', help='Set the log level',
    expose_value=False,
    callback=set_logging,
    type=click.Choice(['DEBUG', 'INFO', 'WARNING', 'ERROR', 'CRITICAL']))
@click.version_option('1.2')
@click.pass_context
def cli(ctx):
    """Eventboard releases script"""
    # Setup the API
    config = ConfigParser.ConfigParser()
    config.read([
        os.path.expanduser('~/.config/octoeb'),
        os.path.expanduser('~/.octoebrc'),
        '.octoebrc'
    ])

    try:
        validate_config(config)
    except Exception as e:
        sys.exit('ERROR: {}'.format(e.message))

    ctx.obj = {
        'mainline': GitHubAPI(
            config.get('repo', 'USER'),
            config.get('repo', 'TOKEN'),
            config.get('repo', 'OWNER'),
            config.get('repo', 'REPO')
        ),
        'fork': GitHubAPI(
            config.get('repo', 'USER'),
            config.get('repo', 'TOKEN'),
            config.get('repo', 'FORK'),
            config.get('repo', 'REPO')
        ),
        'jira': JiraAPI(
            config.get('bugtracker', 'BASE_URL'),
            config.get('bugtracker', 'USER'),
            config.get('bugtracker', 'TOKEN'),
            config.items('bugtracker')
        )
    }

octoeb (pieces)

# setup.py

from setuptools import find_packages
from setuptools import setup


setup(
    name='octoeb',
    version='1.3',
    packages=find_packages(),
    include_package_data=True,
    install_requires=[
        'click',
        'python-slugify',
        'requests',
    ],
    entry_points={
        'console_scripts': [
            'octoeb=octoeb.cli:cli'
        ]
    },
)

Not public yet :(

Coming Soon?

Text

When I use my own CLI

My cowokers using my CLI

No more fat fingers

Cheat sheet

  • -h/--help

 

  • subcommands

 

 

 

  • Multiple Args
  • Multiple Options
  • Repeat Options

 

  • Options and Args
CONTEXT_SETTINGS = dict(
    help_option_names=['-h', '--help'])
@click.group(context_settings=CONTEXT_SETTINGS)
@click.argument('args', nargs=-1)

@click.option('--opt1', '-o1', nargs=2)

@click.option('--opt2', '-o2', multiple=True)
@click.group()
def cli(): pass

@cli.command()
def bar(): pass

@cli.command()
def baz(): pass
@click.command()
@click.argument('bars', nargs=-1)
@click.option('--baz', '-b', default='fuzz')
def foo(bars, baz):
    click.echo('bars = {}'.format(bars))
    click.echo('baz = {}'.format(baz))
foo a b c -o bar
foo -h
foo --help
foo bar 1 2 3
foo baz 1 2 3
foo a b c -o1 10 20 -o2 x -o2 y

Thanks!

Join Us, We're Hiring! 

https://eventboard.io/careers

CLIs for Fun and Profit with Python

By Lucas Roesler

CLIs for Fun and Profit with Python

Python, being the wonderful batteries included language that it is, bundles argparse in the standard library. Beyond argparse we also discuss click and hug. Presented for the SLCPython Meetup group (http://slcpy.com/).

  • 698