Meta fun with tests

Business pb

<form>

New Corporation

devs + lawyers

- total_capital:
    type: int
    formula: "sum(founders__capital)"

- founders__capital_percentage:
    type: [int]
    formula: "[_ / total_capital for _ in founders__capital]"

Processing rules

=> by lawyers!!

DSL in yaml + py (ast)

Human risks...

- total_capital:
    type: int
    forula: "sum(foundrs__capital)"

=> syntax checks!

=> tests!!

Like pyflakes, pylint...

unittest.TestCase

TestCase - basic

- total_capital:
    typ: int
    formula: "sum(foundrs__capital)"

- founders__capital_percentage:
    type: [int]
    formula: "[_ / total_capital for _ in founders__capital]"

- president__birth_day:
    type: int
    formula: "president__birth_date.split('/')[0]"

- president__birth_month:
    type: str
    formula: "month_of_date(president__birth_date)"

- president__birth_year:
    type: int
    formula: "president__birth_date.split('/')[2]"

rules = load_processing_rules()

# >>> rules = {
# >>>     founders__capital: {
# >>>         'type': 'int',
# >>>         'formula': "sum(founders__capital)"
# >>>     ...


class TestRule(TestCase):

    def check_rule_syntax(self, rule):
        return (
            'type' in rule[1].keys() and
            'formula' in rule[1].keys()
        )

    def test_rule1(self):
        self.assertTrue(self.check_rule_syntax(
             rules.items()[0]))

    def test_rule2(self):
        self.assertTrue(self.check_rule_syntax(
             rules.items()[1]))

    def test_rule3(self):
        self.assertTrue(self.check_rule_syntax(
             rules.items()[2]))

    def test_rule4(self):
        self.assertTrue(self.check_rule_syntax(
             rules.items()[3]))

    def test_rule5(self):
        self.assertTrue(self.check_rule_syntax(
             rules.items()[4]))

x5

TestCase - loop

- total_capital:
    typ: int
    formula: "sum(foundrs__capital)"

- founders__capital_percentage:
    type: [int]
    formula: "[_ / total_capital for _ in founders__capital]"

- president__birth_day:
    type: int
    formula: "president__birth_date.split('/')[0]"

- president__birth_month:
    type: str
    formula: "month_of_date(president__birth_date)"

- president__birth_year:
    type: int
    formula: "president__birth_date.split('/')[2]"

rules = load_processing_rules()

# >>> rules = {
# >>>     founders__capital: {
# >>>         'type': 'int',
# >>>         'formula': "sum(founders__capital)"
# >>>     ...


class TestRule(TestCase):

    def check_rule_syntax(self, rule):
        return (
            'type' in rule[1].keys() and
            'formula' in rule[1].keys()
        )

    def test_rules(self):
        for rules in rules:
            self.assertTrue(
                self.check_rule_syntax(rule))

TestCase - parametrized

 
class TestFormula(TestCase):
    ...

     
    def check_syntax(self, rule):
        """ rule is a 2ple like ``(varname, {'type': ..., 'formula': ...})``
        """
        self.assertEqual(len(rule), 2)
        self.assertTrue('formula' in rule[1],
                        msg="Rule doesn't contain a formula")
        self.assertTrue('type' in rule[1],
                        msg="Rule doesn't contain a type")


 
class TestFormula(TestCase):
    ...

     
    def check_syntax(self, rule):
        """ rule is a 2ple like ``(varname, {'type': ..., 'formula': ...})``
        """
        self.assertEqual(len(rule), 2)
        self.assertTrue('formula' in rule[1],
                        msg="Rule doesn't contain a formula")
        self.assertTrue('type' in rule[1],
                        msg="Rule doesn't contain a type")

     
    def check_inputs_exist(self, rule):
        """ In the rule formula, are the expected inputs really available?
        Launched on every varname in the right order.
        """
        formula = rule[1]['formula']
        formula_input_vars = undefined_vars(formula)

        # All the availabe varnames: augmented w/ previously calculated varnames
        available_vars = set(self._all_vars) | _BASE_VARS_SET

        self.assertTrue(formula_input_vars.issubset(available_vars),
                        msg=("Formula expects vars that don't exist: %s"
                             % ', '.join(formula_input_vars - available_vars)))

        # Add the calculated varname
        self._all_vars.append(rule[0])

@class_dec(RULESETS)
class TestFormula(TestCase):
    ...

    @to_tesst('rule syntax is well-formed')
    def check_syntax(self, rule):
        """ rule is a 2ple like ``(varname, {'type': ..., 'formula': ...})``
        """
        self.assertEqual(len(rule), 2)
        self.assertTrue('formula' in rule[1],
                        msg="Rule doesn't contain a formula")
        self.assertTrue('type' in rule[1],
                        msg="Rule doesn't contain a type")

    @to_tesst('all formula inputs already exist')
    def check_inputs_exist(self, rule):
        """ In the rule formula, are the expected inputs really available?
        Launched on every varname in the right order.
        """
        formula = rule[1]['formula']
        formula_input_vars = undefined_vars(formula)

        # All the availabe varnames: augmented w/ previously calculated varnames
        available_vars = set(self._all_vars) | _BASE_VARS_SET

        self.assertTrue(formula_input_vars.issubset(available_vars),
                        msg=("Formula expects vars that don't exist: %s"
                             % ', '.join(formula_input_vars - available_vars)))

        # Add the calculated varname
        self._all_vars.append(rule[0])

Write once, run 5 test methods

Decorators!

def class_dec(testable_lists):
    """
    decorate a class which contains test methods which need to be parametrized
    with testable_lists
    :param testable_lists: [TestableList]
    """
    def class_decorator(klass):
        test_methods = retrieve_test_methods(klass)
        for testable_list in testable_lists:
            attrs = {'testable_list': testable_list}
            for m in test_methods:
                for i, testable_obj in enumerate(testable_list):
                    decorated_m = currying(testable_obj)(m)
                    decorated_m.__test__ = 1
                    test_name = m.__test_name__.format(
                        pos=i + 1,
                        testable_obj=testable_obj)
                    attrs[test_name] = decorated_m

            tl_test_klass = type("Test_%s" % testable_list.name, (klass,),
                                 attrs)
            setattr(modules[klass.__module__], tl_test_klass.__name__,
                    tl_test_klass)
        return klass
    return class_decorator

Use

- founders__capital_percentage:
    type: [int]
    formla: "[_ / total_capital for _ in founders__capital]"

- total_capital:
    type: int
    formula: "sum([_ for _ in founders__capital])"

- president__birth_day:
    formula: "president__birth_date.split('/')[0]"

- president__birth_month:
    type: str
    formula: "month_of_date(president__birth_date)"

- president__birth_year:
    type: int
    formula: "president__birth_date.split('/')[2]"

FE..F..F........
======================================================================
ERROR: (#1, l31): "all formula inputs already exist" on founders__capital_percentage (formulas_test.test_with_meta.Test_incorporation)
----------------------------------------------------------------------
KeyError: 'formula'

======================================================================
FAIL: (#1, l21): "rule syntax is well-formed" on founders__capital_percentage (formulas_test.test_with_meta.Test_incorporation)
----------------------------------------------------------------------
AssertionError: Rule doesn't contain a formula

======================================================================
FAIL: (#3, l21): "rule syntax is well-formed" on president__birth_day (formulas_test.test_with_meta.Test_incorporation)
----------------------------------------------------------------------
AssertionError: Rule doesn't contain a type

======================================================================
FAIL: (#4, l31): "all formula inputs already exist" on president__birth_month (formulas_test.test_with_meta.Test_incorporation)
----------------------------------------------------------------------
AssertionError: Formula expects vars that don't exist: month_of_date

----------------------------------------------------------------------
Ran 16 tests in 0.002s

FAILED (errors=1, failures=3)

Use + fix

- founders__capital_percentage:
    type: [int]
    formula: "[_ / total_capital for _ in founders__capital]"

- total_capital:
    type: int
    formula: "sum([_ for _ in founders__capital])"

- president__birth_day:
    type: int
    formula: "president__birth_date.split('/')[0]"

- president__birth_month:
    type: str
    formula: "month_of_date(president__birth_date)"

- president__birth_year:
    type: int
    formula: "president__birth_date.split('/')[2]"

.F.....F........
======================================================================
FAIL: (#1, l31): "all formula inputs already exist" on founders__capital_percentage (formulas_test.test_with_meta.Test_incorporation)
----------------------------------------------------------------------
AssertionError: Formula expects vars that don't exist: total_capital

======================================================================
FAIL: (#4, l31): "all formula inputs already exist" on president__birth_month (formulas_test.test_with_meta.Test_incorporation)
----------------------------------------------------------------------
AssertionError: Formula expects vars that don't exist: month_of_date

----------------------------------------------------------------------
Ran 16 tests in 0.002s

FAILED (failures=2)

Use + fix + fix

- total_capital:
    type: int
    formula: "sum([_ for _ in founders__capital])"

- founders__capital_percentage:
    type: [int]
    formula: "[_ / total_capital for _ in founders__capital]"

- president__birth_day:
    type: int
    formula: "president__birth_date.split('/')[0]"

- president__birth_month:
    type: str
    formula: "month_of_date(president__birth_date)"

- president__birth_year:
    type: int
    formula: "president__birth_date.split('/')[2]"

................
----------------------------------------------------------------------
Ran 16 tests in 0.002s

OK

Use parametrized tests?

When?

Lots of similar objects => data processing, functional-style code

Pros

- easy to write

- total control on output

Cons

- updates are more involved

- cross-test-runners?

How?

DIY! github.com/legalstart/formulas/blob/master/param_test/decorators.py

pytest.mark.parametrize

nose-parameterized

Paris.py #10

By lajarre

Made with Slides.com

Paris.py #10

  • 1,689

More from lajarre