Oh My Mustache

How to write / deploy / install a command line tool

#gradle #kotlin #homebrew

Why ?

Stop bash scripts
Better compatibility with OS
Easy to write
Easy to install / update
Easy to use

Goals

Write a command line tool with Kotlin
Build and deploy it on GitHub
Install it via Homebrew
Add auto-completion

Demo subject

Provide Mustache templating via command line

Example

$ oh-my-mustache --template "Hi {{name}} !" --variable name "my Lord"

Hi my Lord !

Code

Write your program with Kotlin

We will need

Gradle : Build the code
Kotlin : JVM language
ArgParser : Kotlin library to parse args
Mustache.java : Java library to compile mustache templates

ekino/oh-my-mustache

git clone https://github.com/ekino/oh-my-mustache

# For each step, jump to commit and discard current changes
git checkout -f STEP-X

Code - Step 1

Init project

# Jump to STEP-1 and discard changes
git checkout -f STEP-1

Simple Hello World

Init a simple gradle project
Add the application plugin
Add the kotlin plugin
Display Hello World

build.gradle

buildscript {
  // ...

  dependencies {
    classpath "org.jetbrains.kotlin:kotlin-gradle-plugin:$kotlin_version"
  }
}

plugins {
  id 'org.jmailen.kotlinter' version '1.7.0'
}

apply plugin: 'application'
apply plugin: 'kotlin'
apply plugin: 'java'

// ...

dependencies {

  compile "org.jetbrains.kotlin:kotlin-stdlib:$kotlin_version"
  compile "org.jetbrains.kotlin:kotlin-reflect:$kotlin_version"

  // ...
}

mainClassName = "com.ekino.oss.tooling.demo.MainKt"

// ...

Main.kt

package com.ekino.oss.tooling.demo

import java.io.OutputStreamWriter
import kotlin.system.exitProcess

fun main(args: Array<String>) {

    displayAndExit("Hello world !")
}

fun displayAndExit(message: String = "", exitCode: Int = 0) {
    val writer = OutputStreamWriter(if (exitCode == 0) System.out else System.err)
    writer.write("$message\n")
    writer.flush()
    exitProcess(exitCode)
}

build, install, run

$ ./gradlew build install

$ tree build/distributions build/install

build/distributions
├── oh-my-mustache-1.0.0-SNAPSHOT.tar
└── oh-my-mustache-1.0.0-SNAPSHOT.zip
build/install
└── oh-my-mustache
    ├── bin
    │   ├── oh-my-mustache
    │   └── oh-my-mustache.bat
    └── lib
        ├── annotations-13.0.jar
        ├── kotlin-reflect-1.2.30.jar
        ├── kotlin-stdlib-1.2.30.jar
        └── oh-my-mustache-1.0.0-SNAPSHOT.jar

$ ./build/install/oh-my-mustache/bin/oh-my-mustache

Hello world !

Code - Step 2

Feature implementation

# Jump to STEP-2 and discard changes
git checkout -f STEP-2

Template

Fill template with data

build.gradle

compile "com.github.spullara.mustache.java:compiler:$mustache_version"

gradle.properties

mustache_version = 0.8.18

Main.kt

fun main(args: Array<String>) {

    val templateReader = StringReader("Hello {{name}} !")
    val context = mapOf("name" to "world")
    transformAndPrint(templateReader, context, OutputStreamWriter(System.out))
    exit()
}

fun transformAndPrint(template: Reader, context: Map<String, Any>, writer: Writer) {
    val mf = DefaultMustacheFactory()
    val mustache = mf.compile(template, "template")
    mustache.execute(writer, context)
    writer.flush()
}

TemplatingTest.kt

@DisplayName("Tests about mustache templating")
class TemplatingTest {

    @Test
    @DisplayName("Should convert simple template from String")
    fun should_convert_simple_template_from_string() {

        // Given
        val template = """
                Hi {{name}} !
            """.trimIndent()
        val reader = StringReader(template)
        val context = mapOf("name" to "John")

        // When
        val writer = StringWriter()
        transformAndPrint(reader, context, writer)

        // Then
        writer.toString() shouldBe "Hi John !"
    }

    @Test
    @DisplayName("Should convert template from File")
    fun should_convert_template_from_file() {

        // Given
        val reader = FileReader("/template.mustache".resourceFile())
        val context = mapOf(
                "contact" to mapOf(
                        "firstname" to "John",
                        "lastname" to "Doe"
                ),
                "events" to listOf(
                        mapOf("title" to "Meeting 1"),
                        mapOf("title" to "Meeting 2")
                )
        )

        // When
        val writer = StringWriter()
        transformAndPrint(reader, context, writer)

        // Then
        writer.toString() shouldBe """
Hi John Doe !
You have these events today :
    - Meeting 1
    - Meeting 2
""".trimStart()
    }
}

Test utils

object Utils {

    fun loadResource(path: String) = Utils::class.java.getResource(path)!!
}

fun String.resourcePath() = Utils.loadResource(this).path!!

fun String.resourceFile() = File(resourcePath())

Code - Step 3

Command line args

# Jump to STEP-3 and discard changes
git checkout -f STEP-3

Parse arguments

Example - Main.kt

// This is free and unencumbered software released into the public domain.
//
// Anyone is free to copy, modify, publish, use, compile, sell, or
// distribute this software, either in source code form or as a compiled
// binary, for any purpose, commercial or non-commercial, and by any
// means.
//
// In jurisdictions that recognize copyright laws, the author or authors
// of this software dedicate any and all copyright interest in the
// software to the public domain. We make this dedication for the benefit
// of the public at large and to the detriment of our heirs and
// successors. We intend this dedication to be an overt act of
// relinquishment in perpetuity of all present and future rights to this
// software under copyright law.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
// EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
// MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
// IN NO EVENT SHALL THE AUTHORS BE LIABLE FOR ANY CLAIM, DAMAGES OR
// OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
// ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
// OTHER DEALINGS IN THE SOFTWARE.
//
// For more information, please refer to <http://unlicense.org/>

package com.xenomachina.argparser.example

import com.xenomachina.argparser.ArgParser
import com.xenomachina.argparser.SystemExitException
import com.xenomachina.argparser.default
import com.xenomachina.argparser.mainBody
import java.io.File

enum class OptimizationMode { GOOD, FAST, CHEAP }

/**
 * These are the arguments to our program. Each of the properties uses a delegate from an ArgParser, which maps it to
 * option(s) or positional argument(s).
 */
class ExampleArgs(parser: ArgParser) {
    val verbose by parser.flagging("-v", "--verbose",
            help = "enable verbose mode")

    val name by parser.storing("-N", "--name",
            help = "name of the widget").default("John Doe")

    val size by parser.storing("-s", "--size",
            help = "size of the plumbus") { toInt() }

    val includeDirs by parser.adding("-I",
            help = "directory to search for header files") { File(this) }

    val optimizeFor by parser.mapping(
            "--good" to OptimizationMode.GOOD,
            "--fast" to OptimizationMode.FAST,
            "--cheap" to OptimizationMode.CHEAP,
            help = "what to optimize for")

    val sources by parser.positionalList("SOURCE",
            help = "source filename",
            sizeRange = 1..Int.MAX_VALUE)

    val destination by parser.positional("DEST",
            help = "destination filename")
}

/**
 * The main function of our program calls mainBody, which will handle any
 * [SystemExitException] thrown by the [ArgParser] or its delegates. This
 * includes displaying `--help` as well as error messages, and exiting the
 * process with an appropriate status code.
 */
fun main(args: Array<String>) = mainBody {
    // We construct an ArgParser, passing it unparsed command-line arguments,
    // args. Its parseInto method will instantiate ExampleArgs (passing in
    // this ArgParser), and then force parsing to occur. The resulting
    // ExampleArgs instance contains our parsed arguments.
    val parsedArgs = ArgParser(args).parseInto(::ExampleArgs)

    // At this point our parsed arguments are ready use.
    parsedArgs.run {
        println("""
                verbose =     $verbose
                name =        $name
                size =        $size
                includeDirs = $includeDirs
                optimizeFor = $optimizeFor
                sources =     $sources
                destination = $destination""".trimIndent())
    }
}

Example - Usage

$ ./kotlin-argparser-example --fast --size 5 here there everywhere
verbose =     false
name =        John Doe
size =        5
includeDirs = []
optimizeFor = FAST
sources =     [here, there]
destination = everywhere

Example - Generated help

$ ./kotlin-argparser-example --help
usage: ./kotlin-argparser-example [-h] [-v] [-N NAME] -s SIZE [-I I]... --good
                                  SOURCE... DEST

required arguments:
  -s SIZE,      size of the plumbus
  --size SIZE

  --good,       what to optimize for
  --fast,
  --cheap


optional arguments:
  -h, --help    show this help message and exit

  -v,           enable verbose mode
  --verbose

  -N NAME,      name of the widget
  --name NAME

  -I I          directory to search for header files


positional arguments:
  SOURCE        source filename

  DEST          destination filename

build.gradle

compile "com.xenomachina:kotlin-argparser:$kotlin_argparser_version"
compile "com.fasterxml.jackson.core:jackson-databind:$jackson_version"
compile "com.fasterxml.jackson.module:jackson-module-kotlin:$jackson_version"
compile "com.natpryce:konfig:$konfig_version"

gradle.properties

kotlin_argparser_version = 2.0.4
jackson_version = 2.9.3
konfig_version = 1.6.1.0

Args model

class MainArgs(parser: ArgParser) {

    val verbose by parser.flagging(
            "--verbose",
            help = "enable the verbose mode"
    )

    val version by parser.flagging(
            "--version",
            help = "display the version and exit"
    )

    val contextFile by parser.storing(
            "--context-file",
            help = "context file path") {
        File(this)
    }
            .default<File?>(null)
            .addValidator {
                value?.let {
                    if (!it.exists()) throw InvalidArgumentException("Unable to find file at path ${it.path}")
                }
            }

    val simpleContext by parser.option<MutableMap<String, String>>(
            "-v", "--variable",
            argNames = listOf("NAME", "VALUE"),
            isRepeating = true,
            help = "simple variable to add to context") {
        value.orElse { mutableMapOf<String, String>() }.apply { put(arguments[0], arguments[1]) }
    }
            .default(mutableMapOf<String, String>())

    val templateFile by parser.storing(
            "--template-file",
            help = "template file path") {
        File(this)
    }
            .default<File?>(null)
            .addValidator {
                value?.let {
                    if (!it.exists()) throw InvalidArgumentException("Unable to find file at path ${it.path}")
                }
            }

    val templateText by parser.storing(
            "-t", "--template",
            help = "template text")
            .default<String?>(null)
}

Args parsing

fun main(args: Array<String>) = mainBody {

    val parsedArgs = ArgParser(args).parseInto(::MainArgs)

    if (parsedArgs.verbose) {
        println("Verbose mode enabled")
        println("Input parameters : ${args.toList()}\n")
    }

    if (parsedArgs.version) {
        exit("${Application.name} version ${Application.config[project.version]}")
    }

    val context = mutableMapOf<String, Any>()
    parsedArgs.contextFile
            ?.let { jacksonObjectMapper().readValue<Map<String, Any>>(it) }
            ?.let { context.putAll(it) }

    parsedArgs.simpleContext
            .let { context.putAll(it) }

    val templateReader = parsedArgs.templateFile
            ?.let { FileReader(it) }
            ?: parsedArgs.templateText
                    ?.let { StringReader(it) }
            ?: throw InvalidArgumentException("No template input")

    transformAndPrint(templateReader, context, OutputStreamWriter(System.out))
    exit()
}

Application.kt

class Application {

    companion object {
        const val name = "oh-my-mustache"
        val config by lazy {
            systemProperties() overriding
                    EnvironmentVariables()
            ConfigurationProperties.fromResource("config.properties")
        }
    }
}

object project : PropertyGroup() {
    val version by stringType
}

config.properties

project.version=${project-version}

Test command line output

    @Test
    @DisplayName("should display help")
    fun should_display_help() {

        runCommandLine("--help") { out: String, err: String ->
            out shouldBe """
usage: [-h] [--verbose] [--version] [--context-file CONTEXT_FILE]
       [-v NAME VALUE]... [--template-file TEMPLATE_FILE] [-t TEMPLATE]

optional arguments:
  -h, --help                      show this help message and exit

  --verbose                       enable the verbose mode

  --version                       display the version and exit

  --context-file CONTEXT_FILE     context file path

  -v NAME VALUE,                  simple variable to add to context
  --variable NAME VALUE

  --template-file TEMPLATE_FILE   template file path

  -t TEMPLATE,                    template text
  --template TEMPLATE

""".trimStart()
            err shouldBe empty()
        }
    }


    @Test
    @DisplayName("should use context from json file and override with variable")
    fun should_use_context_from_json_file_and_override_with_variable() {

        runCommandLine(
                "--template-file", "/template.mustache".resourcePath(),
                "--context-file", "/context.json".resourcePath(),
                "--variable", "contact.lastname", "Wayne"
        ) { out: String, err: String ->
            out shouldBe """
Hi John Wayne !
You have these events today :
    - Meeting 1
    - Meeting 2

""".trimStart()
            err shouldBe empty()
        }
    }

Test command line output

template.mustache

Hi {{contact.firstname}} {{contact.lastname}} !
You have these events today :
{{#events}}
    - {{title}}
{{/events}}

context.json

{
  "contact": {
    "lastname": "Doe",
    "firstname": "John"
  },
  "events": [
    {
      "title": "Meeting 1"
    },
    {
      "title": "Meeting 2"
    }
  ]
}

Release - Step 4

Build and deploy

# Jump to STEP-4 and discard changes
git checkout -f STEP-4

Release your tool

Source code on GitHub
Build on Travis
Store binaries on GitHub Releases

.travis.yml

language: java

jdk:
  - oraclejdk8

# cache between builds
cache:
  directories:
  - $HOME/.m2
  - $HOME/.gradle

deploy:
  # Github deploy on TAG
  - provider: releases
    api_key:
      secure: ******************
    file_glob: true
    file: "$TRAVIS_BUILD_DIR/build/distributions/*"
    skip_cleanup: true
    body: "Release of $TRAVIS_TAG"
    draft: false
    on:
      tags: true

GitHub Releases

Install - Step 5

Manual vs Auto install

# Jump to STEP-5 and discard changes
git checkout -f STEP-5

Manual install

Go to GitHub Releases
Download / unarchive
Find the executable in the bin directory
Create a symlink to it :

ln -s /[PATH_TO_BIN_DIR]/oh-my-mustache /usr/local/bin/oh-my-mustache

Auto install

Homebrew

Available on Mac OS : Homebrew
Available on Linux : Linuxbrew
Create a "tap"
- Git repository
- Store unofficial formulas
- ekino/homebrew-formulas
Create a Formula
Install the tool

oh-my-mustache.rb

class OhMyMustache < Formula
  desc "Templating with Mustache"
  homepage "https://github.com/ekino/oh-my-mustache"
  url "https://github.com/ekino/oh-my-mustache/releases/download/1.0.0/oh-my-mustache-1.0.0.zip"
  sha256 "7fd7c482a62f614e6c45ed5e705d6abf81761cdbfb239708e4dabc071ea0091c"
  head "https://github.com/ekino/oh-my-mustache"

  def install
    libexec.install "bin"
    libexec.install "lib"
    bin.install_symlink "#{libexec}/bin/oh-my-mustache" => "oh-my-mustache"
  end

  test do
    system bin/"oh-my-mustache", "-t", "Hi {{name}} !", "-v", "name", "brew user"
  end
end

Install it !

$ brew tap ekino/formulas
...
$ brew install oh-my-mustache
...

Completion - Step 6

Auto completion for Bash and Zsh

# Jump to STEP-6 and discard changes
git checkout -f STEP-6

Bash completion

Introduction to bash completion
Write the Bash script
Apply it

ohmymustache-completion.bash

#!/usr/bin/env bash

_oh-my-mustache() {

    local DEFAULT_ARGS=("--help -h --verbose --version")
    local TEMPLATE_ARGS=("--template-file --template -t")
    local CONTEXT_ARGS=("--context-file --variable -v")

    COMPREPLY=()
    local cur=${COMP_WORDS[COMP_CWORD]}
    local prev=${COMP_WORDS[COMP_CWORD-1]}

    case "$prev" in
        "--context-file"|"--template-file")
            COMPREPLY=($(compgen -f ${cur}))
            return 0
        ;;
    esac

    COMPREPLY=($(compgen -W "${DEFAULT_ARGS} ${TEMPLATE_ARGS} ${CONTEXT_ARGS}" -- ${cur}))
    return 0
}

complete -F _oh-my-mustache oh-my-mustache

Try it

cd bin/
source ../completion/ohmymustache-completion.bash

Zsh completion

How to
Bash completion compatibility
- Article (dated 05/01/2010)
- May need to add :

autoload -U compinit && compinit
autoload -U bashcompinit && bashcompinit

Install completion

using Homebrew

oh-my-mustache.rb

class OhMyMustache < Formula
  desc "Templating with Mustache"
  homepage "https://github.com/ekino/oh-my-mustache"
  url "https://github.com/ekino/oh-my-mustache/releases/download/1.0.0/oh-my-mustache-1.0.0.zip"
  sha256 "7fd7c482a62f614e6c45ed5e705d6abf81761cdbfb239708e4dabc071ea0091c"
  head "https://github.com/ekino/oh-my-mustache"

  depends_on "bash-completion" => :recommended

  def install
    libexec.install "bin"
    libexec.install "lib"
    bin.install_symlink "#{libexec}/bin/oh-my-mustache" => "oh-my-mustache"

    bash_completion.install "completion/ohmymustache-completion.bash"
  end

  def caveats; <<~EOS
    To use completion, add the following line to your ~/.bash_profile or ~/.zshrc:
      [ -f #{etc}/bash_completion.d/ohmymustache-completion.bash ] && source #{etc}/bash_completion.d/ohmymustache-completion.bash

    /!\\ ZSH users : you may need to add these two lines before the previous one
      autoload -U compinit && compinit
      autoload -U bashcompinit && bashcompinit
    EOS
  end

  test do
    system bin/"oh-my-mustache", "-t", "Hi {{name}} !", "-v", "name", "brew user"
  end
end

Result

$ brew tap ekino/formulas
==> Tapping ekino/formulas
Cloning into '/usr/local/Homebrew/Library/Taps/ekino/homebrew-formulas'...
remote: Counting objects: 6, done.
remote: Compressing objects: 100% (5/5), done.
remote: Total 6 (delta 0), reused 6 (delta 0), pack-reused 0
Unpacking objects: 100% (6/6), done.
Tapped 1 formula (31 files, 27.9KB)

$ brew install oh-my-mustache
==> Installing oh-my-mustache from ekino/formulas
==> Downloading https://github.com/ekino/oh-my-mustache/releases/download/1.0.0/oh-my-mustache-1.0.0.zip
Already downloaded: /Users/leomillon/Library/Caches/Homebrew/oh-my-mustache-1.0.0.zip
==> Caveats
To use completion, add the following line to your ~/.bash_profile or ~/.zshrc:
  [ -f /usr/local/etc/bash_completion.d/ohmymustache-completion.bash ] && source /usr/local/etc/bash_completion.d/ohmymustache-completion.bash

/!\ ZSH users : you may need to add these two lines before the previous one
  autoload -U compinit && compinit
  autoload -U bashcompinit && bashcompinit

Bash completion has been installed to:
  /usr/local/etc/bash_completion.d
==> Summary
🍺  /usr/local/Cellar/oh-my-mustache/1.0.0: 19 files, 7.5MB, built in 1 second

Conclusion

Demo

Feedbacks

That was cool !
- Fast and fun to write with Kotlin
- Easy to share / install with Homebrew
- Pleasant to use with completion
- Feels like a "complete" tool
But ...
- ArgParser not flexible
- Completion is "static" and "bash first"
- Performance may be a problem

Another example

leomillon/jwtctl

Why ?

Goals

Demo subject

Provide Mustache templating via command line

Example

Code

Write your program with Kotlin

We will need

ekino/oh-my-mustache

Code - Step 1

Init project

Simple Hello World

build.gradle

Main.kt

build, install, run

Code - Step 2

Feature implementation

Template

Fill template with data

build.gradle

gradle.properties

Main.kt

TemplatingTest.kt

Test utils

Code - Step 3

Command line args

Parse arguments

Example - Main.kt

Example - Usage

Example - Generated help

build.gradle

gradle.properties

Args model

Args parsing

Application.kt

config.properties

Test command line output

Test command line output

template.mustache

context.json

Release - Step 4

Build and deploy

Release your tool

.travis.yml

GitHub Releases

Install - Step 5

Manual vs Auto install

Manual install

Auto install

Homebrew

Homebrew

oh-my-mustache.rb

Install it !

Completion - Step 6

Auto completion for Bash and Zsh

Bash completion

ohmymustache-completion.bash

Try it

Zsh completion

Install completion

using Homebrew

oh-my-mustache.rb

Result

Conclusion

Demo

Feedbacks

Another example

Thank you

and write cool tools!