Image Source: pytest-dev

Build Plugins

with Pluggy

__init__

  • I'm Kracekumar from Bangalore.

 

  • Software Engineer.

 

  • Likes reading Fiction.

Talk Overview

  • Why develop as plugins?

 

  • What is a Plugin system (pluggy)?

 

  • How pluggy works?

Why develop as plugins?

Normal Code

from formatter import json_formatter, tabulate


def render_output(output, format):
    if format == "json":
        print(json_formatter(output))
    elif format == "table":
        print(tabulate(output))


class Application:
    def run(self):
        self.get_input()
        term = self.processor.process(self.data)
        output = self.search(term)
        render_output(output, self.format)

Short Comings

  • Needs new release for every output format

 

  • Difficult to extend support for all user requests

 

  • Not extensible

 

 

Plugin System

Plugin

noun: plug-in is a software component that adds a specific feature to an existing computer program.

Example: Google Chrome Extension

What is pluggy?

Pluggy provides a structured way to manage, discover plugins, and enable hooks to change the behavior of the host program at runtime.

How pluggy works?

Plugin Working

$tree                                                                                                                                                                                               (pluggy_talk)
.
├── LICENSE
├── README.md
├── hookspecs.py
├── host.py
├── output.py
├── requirements.txt
└── tests.py

Terminology

  • Host Program/Core system (host.py)
  • Plugin (output.py)
  • Plugin Manager (instance in host.py)
  • Hook Specification (hookspec.py)
  • Hook Implementation (function/method in output.py)
$cat host.py
# imports

class Search:
    def __init__(self, term, hook, kwargs):
        # initializes the attrs

    def make_request(self):
        # makes the request to gutenberg URL

    def run(self):
        # co-ordinates the flow

def get_plugin_manager():
    # plugin spec, implementation registration

@click.group()
def cli():
    pass


@cli.command()
# click options
def search(title, author, **kwargs):
    # validates the user input, manages search workflow

def setup():
    pm = get_plugin_manager()
    pm.hook.get_click_group(group=cli)

if __name__ == "__main__":
    setup()
    cli()

Details

1. Hookspec

# hookspec.py
import pluggy

hookspec = pluggy.HookspecMarker(project_name="gutenberg")


@hookspec
def print_output(resp, config):
    """Print formatted output"""

2. Implementation

# Name should match hookspec marker
hookimpl = pluggy.HookimplMarker(project_name="gutenberg")


@hookimpl
def print_output(resp, config):
    """Print output"""
    data = resp.json()
    table = [
        {
            "name": result["authors"][0]["name"],
            "bookshelves": result["bookshelves"],
            "copyright": result["copyright"],
            "download_count": result["download_count"],
            "title": result["title"],
            "media_type": result["media_type"],
            "xml": result["formats"]["application/rdf+xml"],
        }
        for result in data["results"]
    ]
    indent = config.get("indent", 4)
    if config.get('format', '') == 'json':
    	print(f"Using the indent size as {indent}")
    	formatted_json = json.dumps(table, sort_keys=True, 
                                    indent=indent)
    	if config.get('colorize'):
            print(colorize(formatted_json))
        else:
            print(formatted_json)

3. Plugin Manager

import hookspecs
import output


def get_plugin_manager():
    pm = pluggy.PluginManager(project_name="gutenberg")
    pm.add_hookspecs(hookspecs)
    # Add a Python file
    pm.register(output)
    # Or add a load from setuptools entrypoint
    pm.load_setuptools_entrypoints("gutenberg")
    return pm

4. Invoke the hook

class Search:
    def __init__(self, term, hook, kwargs):
        self.term = term
        self.hook = hook
        self.kwargs = kwargs

    def make_request(self):
        resp = requests.get(f"http://gutendex.com/books/?search={self.term}")
        return resp

    def run(self):
        resp = self.make_request()
        self.hook.print_output(resp=resp, config=self.kwargs)


@cli.command()
@click.option("--title", "-t", type=str, help="Title to search")
@click.option("--author", "-a", type=str, help="Author to search")
def search(title, author, **kwargs):
    if not (title or author):
        print("Pass either --title or --author")
        exit(-1)
    else:
        pm = get_plugin_manager()
        search = Search(title or author, pm.hook, kwargs)
        search.run()

Example Output

Pluggy Internals

Calling Hooks

Hook working

 

  • pm.hook.print_output - 1:N calls

 

  • firstresult=True

Testing The Plugin

def test_print_output(capsys):
    resp = requests.get("http://gutendex.com/books/?search=Kafka")
    print_output(resp, {})

    captured = capsys.readouterr()
    assert len(json.loads(captured.out)) >= 1

Unit Test

def test_search():
    setup()
    runner = CliRunner()
    result = runner.invoke(
        search,
        ["-t", "My freedom and My bondage", 
        "--indent", 8, "--colorize", "false"],
    )

    expected_output = """
[
        {
                "bookshelves": [
                        "African American Writers",
                        "Slavery"
                ],
                "copyright": false,
                "download_count": 1201,
                "media_type": "Text",
                "name": "Douglass, Frederick",
                "title": "My Bondage and My Freedom",
                "xml": "http://www.gutenberg.org/ebooks/202.rdf"
        }
]
    """
    assert result
    assert result.output.strip() == expected_output.strip()

Integration Test

Real World Projects

 

 

Useful Link

 

Stay Safe, Wear Mask!

 

Thank You.

Made with Slides.com