Conftest: Expressing and Enforcing Policy for Configuration Systems

Anthony Critelli

About Me

  • Current SA at Cloudify, all-around Linux guy, recovering network engineer
  • Adjunct at RIT
  • Core contributor to Red Hat Enable SysAdmin blog
  • Maintainer of Rucksack, a Python
    tool for storing one-liners
  • City of Rochester (Swillburg) resident

You don't need a picture of me because I'm standing right in front of you, so here are my dogs: Kramer and Newman

Problem Space

Automation and IaC is wonderful, but...

hosts: all
tasks:
  - name: Set file permissions
    ansible.builtin.file:
      path: /etc/passwd
      mode: 0777
  - name: Install very important package
    ansible.builtin.package:
      name: telnet
      state: latest

1

Write config with policy violation

2

Perform code review. Human may or may not notice bad practice

3

Policy violation may or may not make it into production

  • How do we add guardrails to IaC without slowing down velocity?
  • "Just do code review" is a tempting response
    • Impractical, slow, and error-prone
    • Fails to integrate continuous learning
  • Every tool, from Ansible to K8s, has its own linter/policy enforcement tool/etc.
    • Cognitive difficulty in learning different tools
  • Lack of policy language portability across teams
    • Security team - interested in security policy
    • Architecture/systems team - interested in policy related to capacity planning
    • Etc.
  • Conftest allows us to express policy in a common language

 

A common policy language

Intro to OPA and Rego

Open Policy Agent

  • A "unified toolset and framework for policy across the cloud native stack."
  • Write policy in a declarative language
    • Decoupled from app or service
    • App or service then requests policy decisions from OPA
  • Architecturally either a library or a daemon (HTTP service)
  • Rego - query language used by OPA
    • Based on Datalog

Rego Basics - Types

# Scalar
admin_user := "bob"

# List
packages := [ "telnet", "netcat" ]

# Set
apps_set := { "telnet", "netcat", "ping", "tcpdump" }

# List of objects
users := [
  {
    "name": "alice",
    "admin": true,
    "account_enabled": false
  },
  {
    "name": "bob",
    "admin": true,
    "account_enabled": true
  },
  {
    "name": "trudy",
    "admin": false,
    "account_enabled": true
  }
]

Rego Basics - Complete Rules

package main
import future.keywords.in
import future.keywords.if

# Constants are technically rules
apps := [ "telnet", "netcat", "ping", "tcpdump" ]

# The rules below are equivalent

# Uses newer keywords "if" and "in"
contains_netcat := true if {
    some app in apps
    app == "netcat"
}

# If the head doesn't define a value, then it defaults to true
contains_netcat {
    apps[_] == "netcat"
}

Rego Basics - Partial Rules

package main
import future.keywords.in
import future.keywords.if
import future.keywords.contains

apps := [
  { "name": "apache", "latest": false },
  { "name": "java", "latest": true },
  { "name": "vim", "latest": false },
]

outdated_packages contains app.name if {
  some app in apps
  app.latest == false
}

outdated_packages[name] {
  some app in apps
  not app.latest
  name := app.name
}

outdated_apache {
  some app
  outdated_packages[app]
  app == "apache"
}

Conftest

Conftest

  • OPA acts on JSON input
    • Conftest extends this to other formats
    • YAML, TOML, INI, HCL, .env, and more
  • Easily installed
  • Three types of rules
    • deny - Fails the policy evaluation with a message
    • violation - Fails the policy evaluation with the ability to return more complex data
    • warn - Does not fail the policy evaluation, but generates a warning

Installation

$ wget https://github.com/open-policy-agent/conftest/releases/download/v0.34.0/conftest_0.34.0_Linux_x86_64.tar.gz

$ tar -xf conftest_0.34.0_Linux_x86_64.tar.gz 

$ sudo mv conftest /usr/local/bin/conftest

Usage

  1. Define policies in policy/ directory
$ tree
.
├── playbook.yaml
├── policy
│   └── main.rego
└── README.md

1 directory, 3 files

2. Run tests against config

$ conftest test playbook.yaml
FAIL - playbook.yaml - main - Netcat is not a permitted package

1 test, 0 passed, 0 warnings, 1 failure, 0 exceptions

Demo

Packaging and Namespaces

  • Policies can be bundled into artifacts (zip files) and stored in an OCI registry (or other external storage)
    • conftest pull and conftest push
  • Policies can also be divided into namespaces and Conftest can run all policies across multiple namespaces
    • "--all-namespaces" flag
package security.example.com

...

warn_sudo contains msg if {
  some key,val in input.path
  "sudo" in val.capabilities
  msg := sprintf("Sudo capabilities granted on %v", [key])
}

External Data Sources

  • Hard-coding values into policy doesn't seem ideal
  • External data sources (YAML, JSON) files can be passed to conftest
$ conftest test docker-compose.yaml --data data.yaml
FAIL - docker-compose.yaml - main - Port 3306 is not allowed to be exposed.

1 test, 0 passed, 0 warnings, 1 failure, 0 exceptions
import data.allowed_ports

deny_ports contains msg if {
  some service in input.services
  some published_port in service.ports
  port := split(published_port, ":")[0]
  not to_number(port) in allowed_ports
  msg := sprintf("Port %v is not allowed to be exposed.", [port])
}

Ideas and Resources

CI/CD Pipelines

  -o, --output string         Output format for conftest results - valid options are: [stdout json tap table junit github] (default "stdout")

Git pre-commit hooks

  • Automatically run prior to a commit
  • Can gate the commit from ever making it into the branch
  • Requires a script in .git/hooks
    • Conftest must be installed on local system
    • Cross platform concerns

 

$ git commit -m "Add initial Dockerfile"
Running conftest pre-commit hook...
FAIL - Dockerfile - main - Base image 'ubuntu:12.04' is not in list of allowed base images

1 test, 0 passed, 0 warnings, 1 failure, 0 exceptions

Helpful Tips and Resources

Parsing

# Parsed config files can then be used in the Rego playground
$ conftest parse vault_policy.hcl
{
  "path": {
    "auth/token/*": {
      "capabilities": [
        "create",
        "read",
        "update",
        "sudo"
      ]
    },
    "monolithic-database/*": {
      "capabilities": [
        "create",
        "read",
        "update"
      ]
    }
  }
}
  • Parses the config file and outputs the resulting JSON
  • Extremely useful for understanding the data structure passed into policy evaluations

Rego Playground

Useful Articles and Tutorials

Thank you!

Contact: critellia at gee mail dot com

Connect: https://www.linkedin.com/in/anthonycritelli/

Slides: https://slides.com/anthonycritelli/conftest-2022

Examples: https://github.com/acritelli/conftest-talk-2022

Unused Slides

Deny Rules

deny_telnet_with_package contains msg if {
  some task in input.tasks
  task["ansible.builtin.package"].name == "telnet"
  task["ansible.builtin.package"].state != "absent"
  msg := sprintf("Task '%v' contains illegal package telnet", [task["name"]])
}
❯ conftest test playbook.yaml
FAIL - playbook.yaml - main - Task 'Install telnet' contains illegal package telnet

1 test, 0 passed, 0 warnings, 1 failure, 0 exceptions

Violation Rules

violation_netcat contains complex_message if {
  some task in input.tasks
  task["ansible.builtin.package"].name == "netcat"
  task["ansible.builtin.package"].state != "absent"
  complex_message := {
    "msg": "Netcat is not a permitted package",
    "details": {
      "wiki_article": "https://wiki.example.com/allowed_packages",
      "security_policy": "SEC-12345"
    }
  }
}
❯ conftest test playbook.yaml
FAIL - playbook.yaml - main - Netcat is not a permitted package

1 test, 0 passed, 0 warnings, 1 failure, 0 exceptions
 
❯ conftest test playbook.yaml -o json | jq -r .[0].failures
[
  {
    "msg": "Netcat is not a permitted package",
    "metadata": {
      "details": {
        "security_policy": "SEC-12345",
        "wiki_article": "https://wiki.example.com/allowed_packages"
      }
    }
  }
]

Warn Rules

warn_sudo contains msg if {
  some key, val in input.path
  "sudo" in val.capabilities
  msg := sprintf("Sudo capabilities granted on %v", [key])
}

❯ conftest test vault_policy.hcl --all-namespaces
WARN - vault_policy.hcl - security.example.com - Sudo capabilities granted on auth/token/*

1 test, 0 passed, 1 warning, 0 failures, 0 exceptions

1

Write config with policy violation

2

Automatically flag violation with Conftest

3

Remedy policy violation

5

Policy violations never have the chance to make it into production

4

Human review can focus on quality of code, not presence of violations

A better way

Made with Slides.com