Meta fun with tests
Business pb
<form>
New Corporation
devs + lawyers
- total_capital:
type: int
formula: "sum(founders__capital)"
- founders__capital_percentage:
type: [int]
formula: "[_ / total_capital for _ in founders__capital]"
Processing rules
=> by lawyers!!
DSL in yaml + py (ast)
Human risks...
- total_capital:
type: int
forula: "sum(foundrs__capital)"
=> syntax checks!
=> tests!!
Like pyflakes, pylint...
unittest.TestCase
TestCase - basic
- total_capital:
typ: int
formula: "sum(foundrs__capital)"
- founders__capital_percentage:
type: [int]
formula: "[_ / total_capital for _ in founders__capital]"
- president__birth_day:
type: int
formula: "president__birth_date.split('/')[0]"
- president__birth_month:
type: str
formula: "month_of_date(president__birth_date)"
- president__birth_year:
type: int
formula: "president__birth_date.split('/')[2]"
rules = load_processing_rules()
# >>> rules = {
# >>> founders__capital: {
# >>> 'type': 'int',
# >>> 'formula': "sum(founders__capital)"
# >>> ...
class TestRule(TestCase):
def check_rule_syntax(self, rule):
return (
'type' in rule[1].keys() and
'formula' in rule[1].keys()
)
def test_rule1(self):
self.assertTrue(self.check_rule_syntax(
rules.items()[0]))
def test_rule2(self):
self.assertTrue(self.check_rule_syntax(
rules.items()[1]))
def test_rule3(self):
self.assertTrue(self.check_rule_syntax(
rules.items()[2]))
def test_rule4(self):
self.assertTrue(self.check_rule_syntax(
rules.items()[3]))
def test_rule5(self):
self.assertTrue(self.check_rule_syntax(
rules.items()[4]))
x5
TestCase - loop
- total_capital:
typ: int
formula: "sum(foundrs__capital)"
- founders__capital_percentage:
type: [int]
formula: "[_ / total_capital for _ in founders__capital]"
- president__birth_day:
type: int
formula: "president__birth_date.split('/')[0]"
- president__birth_month:
type: str
formula: "month_of_date(president__birth_date)"
- president__birth_year:
type: int
formula: "president__birth_date.split('/')[2]"
rules = load_processing_rules()
# >>> rules = {
# >>> founders__capital: {
# >>> 'type': 'int',
# >>> 'formula': "sum(founders__capital)"
# >>> ...
class TestRule(TestCase):
def check_rule_syntax(self, rule):
return (
'type' in rule[1].keys() and
'formula' in rule[1].keys()
)
def test_rules(self):
for rules in rules:
self.assertTrue(
self.check_rule_syntax(rule))
TestCase - parametrized
class TestFormula(TestCase):
...
def check_syntax(self, rule):
""" rule is a 2ple like ``(varname, {'type': ..., 'formula': ...})``
"""
self.assertEqual(len(rule), 2)
self.assertTrue('formula' in rule[1],
msg="Rule doesn't contain a formula")
self.assertTrue('type' in rule[1],
msg="Rule doesn't contain a type")
class TestFormula(TestCase):
...
def check_syntax(self, rule):
""" rule is a 2ple like ``(varname, {'type': ..., 'formula': ...})``
"""
self.assertEqual(len(rule), 2)
self.assertTrue('formula' in rule[1],
msg="Rule doesn't contain a formula")
self.assertTrue('type' in rule[1],
msg="Rule doesn't contain a type")
def check_inputs_exist(self, rule):
""" In the rule formula, are the expected inputs really available?
Launched on every varname in the right order.
"""
formula = rule[1]['formula']
formula_input_vars = undefined_vars(formula)
# All the availabe varnames: augmented w/ previously calculated varnames
available_vars = set(self._all_vars) | _BASE_VARS_SET
self.assertTrue(formula_input_vars.issubset(available_vars),
msg=("Formula expects vars that don't exist: %s"
% ', '.join(formula_input_vars - available_vars)))
# Add the calculated varname
self._all_vars.append(rule[0])
@class_dec(RULESETS)
class TestFormula(TestCase):
...
@to_tesst('rule syntax is well-formed')
def check_syntax(self, rule):
""" rule is a 2ple like ``(varname, {'type': ..., 'formula': ...})``
"""
self.assertEqual(len(rule), 2)
self.assertTrue('formula' in rule[1],
msg="Rule doesn't contain a formula")
self.assertTrue('type' in rule[1],
msg="Rule doesn't contain a type")
@to_tesst('all formula inputs already exist')
def check_inputs_exist(self, rule):
""" In the rule formula, are the expected inputs really available?
Launched on every varname in the right order.
"""
formula = rule[1]['formula']
formula_input_vars = undefined_vars(formula)
# All the availabe varnames: augmented w/ previously calculated varnames
available_vars = set(self._all_vars) | _BASE_VARS_SET
self.assertTrue(formula_input_vars.issubset(available_vars),
msg=("Formula expects vars that don't exist: %s"
% ', '.join(formula_input_vars - available_vars)))
# Add the calculated varname
self._all_vars.append(rule[0])
Write once, run 5 test methods
Decorators!
def class_dec(testable_lists):
"""
decorate a class which contains test methods which need to be parametrized
with testable_lists
:param testable_lists: [TestableList]
"""
def class_decorator(klass):
test_methods = retrieve_test_methods(klass)
for testable_list in testable_lists:
attrs = {'testable_list': testable_list}
for m in test_methods:
for i, testable_obj in enumerate(testable_list):
decorated_m = currying(testable_obj)(m)
decorated_m.__test__ = 1
test_name = m.__test_name__.format(
pos=i + 1,
testable_obj=testable_obj)
attrs[test_name] = decorated_m
tl_test_klass = type("Test_%s" % testable_list.name, (klass,),
attrs)
setattr(modules[klass.__module__], tl_test_klass.__name__,
tl_test_klass)
return klass
return class_decorator
Use
- founders__capital_percentage:
type: [int]
formla: "[_ / total_capital for _ in founders__capital]"
- total_capital:
type: int
formula: "sum([_ for _ in founders__capital])"
- president__birth_day:
formula: "president__birth_date.split('/')[0]"
- president__birth_month:
type: str
formula: "month_of_date(president__birth_date)"
- president__birth_year:
type: int
formula: "president__birth_date.split('/')[2]"
FE..F..F........
======================================================================
ERROR: (#1, l31): "all formula inputs already exist" on founders__capital_percentage (formulas_test.test_with_meta.Test_incorporation)
----------------------------------------------------------------------
KeyError: 'formula'
======================================================================
FAIL: (#1, l21): "rule syntax is well-formed" on founders__capital_percentage (formulas_test.test_with_meta.Test_incorporation)
----------------------------------------------------------------------
AssertionError: Rule doesn't contain a formula
======================================================================
FAIL: (#3, l21): "rule syntax is well-formed" on president__birth_day (formulas_test.test_with_meta.Test_incorporation)
----------------------------------------------------------------------
AssertionError: Rule doesn't contain a type
======================================================================
FAIL: (#4, l31): "all formula inputs already exist" on president__birth_month (formulas_test.test_with_meta.Test_incorporation)
----------------------------------------------------------------------
AssertionError: Formula expects vars that don't exist: month_of_date
----------------------------------------------------------------------
Ran 16 tests in 0.002s
FAILED (errors=1, failures=3)
Use + fix
- founders__capital_percentage:
type: [int]
formula: "[_ / total_capital for _ in founders__capital]"
- total_capital:
type: int
formula: "sum([_ for _ in founders__capital])"
- president__birth_day:
type: int
formula: "president__birth_date.split('/')[0]"
- president__birth_month:
type: str
formula: "month_of_date(president__birth_date)"
- president__birth_year:
type: int
formula: "president__birth_date.split('/')[2]"
.F.....F........
======================================================================
FAIL: (#1, l31): "all formula inputs already exist" on founders__capital_percentage (formulas_test.test_with_meta.Test_incorporation)
----------------------------------------------------------------------
AssertionError: Formula expects vars that don't exist: total_capital
======================================================================
FAIL: (#4, l31): "all formula inputs already exist" on president__birth_month (formulas_test.test_with_meta.Test_incorporation)
----------------------------------------------------------------------
AssertionError: Formula expects vars that don't exist: month_of_date
----------------------------------------------------------------------
Ran 16 tests in 0.002s
FAILED (failures=2)
Use + fix + fix
- total_capital:
type: int
formula: "sum([_ for _ in founders__capital])"
- founders__capital_percentage:
type: [int]
formula: "[_ / total_capital for _ in founders__capital]"
- president__birth_day:
type: int
formula: "president__birth_date.split('/')[0]"
- president__birth_month:
type: str
formula: "month_of_date(president__birth_date)"
- president__birth_year:
type: int
formula: "president__birth_date.split('/')[2]"
................
----------------------------------------------------------------------
Ran 16 tests in 0.002s
OK
Use parametrized tests?
When?
Lots of similar objects => data processing, functional-style code
Pros
- easy to write
- total control on output
Cons
- updates are more involved
- cross-test-runners?
How?
pytest.mark.parametrize
nose-parameterized
Paris.py #10
By lajarre
Paris.py #10
- 1,665