Hack Bulgaria
github.com/HackBulgaria
Radoslav Georgiev
EuroPython 2017
Sofia, Bulgaria
~ 9 Python / Django courses, preparing people for their first job.
Level of frustration:
1
2
3
You run your code, things blow up.
Example: Text file with datetime intervals & python program that checks if there are any overlapping.
$ head -5 intervals.txt
(2017-06-25 19:27:51, 2017-06-25 19:42:51)
(2017-06-25 19:43:51, 2017-06-25 19:58:51)
(2017-06-25 19:59:51, 2017-06-25 20:14:51)
(2017-06-25 20:15:51, 2017-06-25 20:30:51)
(2017-06-25 21:51:51, 2017-06-25 22:06:51)
$ cat intervals.txt | wc -l
100
$ python overlapping.py intervals.txt
All good.
$ python overlapping.py intervals.txt
There are overlapping intervals.
$ python exceptions.py intervals.txt
Traceback (most recent call last):
File "exceptions.py", line 55, in <module>
main()
File "exceptions.py", line 45, in main
result = run(contents)
File "exceptions.py", line 32, in run
data = [parse_datetime(value) for value in contents]
File "exceptions.py", line 32, in <listcomp>
data = [parse_datetime(value) for value in contents]
File "exceptions.py", line 10, in parse_datetime
return datetime.strptime(value, fmt)
File "/usr/lib/python3.5/_strptime.py", line 510, in _strptime_datetime
tt, fraction = _strptime(data_string, format)
File "/usr/lib/python3.5/_strptime.py", line 343, in _strptime
(data_string, format))
ValueError: time data '(' does not match format '%Y-%m-%d %H:%M:%S'
$ python exceptions.py intervals.txt
Traceback (most recent call last):
File "exceptions.py", line 55, in <module>
main()
File "exceptions.py", line 45, in main
result = run(contents)
File "exceptions.py", line 32, in run
data = [parse_datetime(value) for value in contents]
File "exceptions.py", line 32, in <listcomp>
data = [parse_datetime(value) for value in contents]
File "exceptions.py", line 10, in parse_datetime
return datetime.strptime(value, fmt)
File "/usr/lib/python3.5/_strptime.py", line 510, in _strptime_datetime
tt, fraction = _strptime(data_string, format)
File "/usr/lib/python3.5/_strptime.py", line 343, in _strptime
(data_string, format))
ValueError: time data '(' does not match format '%Y-%m-%d %H:%M:%S'
Very important thing when reading stack traces.
$ python exceptions.py intervals.txt
Traceback (most recent call last):
File "exceptions.py", line 55, in <module>
main()
File "exceptions.py", line 45, in main
result = run(contents)
File "exceptions.py", line 32, in run
data = [parse_datetime(value) for value in contents]
File "exceptions.py", line 32, in <listcomp>
data = [parse_datetime(value) for value in contents]
File "exceptions.py", line 10, in parse_datetime
return datetime.strptime(value, fmt)
File "/usr/lib/python3.5/_strptime.py", line 510, in _strptime_datetime
tt, fraction = _strptime(data_string, format)
File "/usr/lib/python3.5/_strptime.py", line 343, in _strptime
(data_string, format))
ValueError: time data '(' does not match format '%Y-%m-%d %H:%M:%S'
def parse_datetime(value: str) -> datetime:
fmt = '%Y-%m-%d %H:%M:%S'
return datetime.strptime(value, fmt)
def run(contents: str) -> bool:
data = [parse_datetime(value) for value in contents]
result = overlapping_intervals(data)
return result
Constant regressions can make you crazy.
Also known as - reproduce the bug.
import unittest
from overlapping_intervals import run
class OverlappingIntervalsTests(unittest.TestCase):
def test_overlapping_intervals(self):
input = ["(2017-06-25 19:27:51, 2017-06-25 19:42:51)",
"(2017-06-25 19:43:51, 2017-06-25 19:58:51)"]
input = '\n'.join(input)
result = run(input)
self.assertFalse(result)
if __name__ == '__main__':
unittest.main()
E
======================================================================
ERROR: test_overlapping_intervals (__main__.OverlappingIntervalsTests)
----------------------------------------------------------------------
.... traceback here ...
----------------------------------------------------------------------
Ran 1 test in 0.011s
def parse_line(value: str) -> Tuple[datetime, datetime]:
parts = value.split(',')
first = parts[0]
second = parts[1]
# Remove ()
first = first[1:]
second = second[0:len(second) - 1]
return parse_datetime(first), parse_datetime(second)
$ python exceptions.py intervals.txt
Traceback (most recent call last):
File "exceptions.py", line 69, in <module>
main()
File "exceptions.py", line 59, in main
result = run(fname)
File "exceptions.py", line 49, in run
data = [parse_line(value) for value in contents.split('\n')]
File "exceptions.py", line 49, in <listcomp>
data = [parse_line(value) for value in contents.split('\n')]
File "exceptions.py", line 24, in parse_line
return parse_datetime(first), parse_datetime(second)
File "exceptions.py", line 10, in parse_datetime
return datetime.strptime(value, fmt)
File "/usr/lib/python3.5/_strptime.py", line 510, in _strptime_datetime
tt, fraction = _strptime(data_string, format)
File "/usr/lib/python3.5/_strptime.py", line 343, in _strptime
(data_string, format))
ValueError: time data ' 2017-06-25 19:42:51' does not match format '%Y-%m-%d %H:%M:%S'
def parse_line(value: str) -> Tuple[datetime, datetime]:
parts = value.split(',')
first = parts[0]
second = parts[1]
# Remove ()
first = first[1:]
second = second[0:len(second) - 1]
import ipdb; ipdb.set_trace()
return parse_datetime(first), parse_datetime(second)
pip install ipdb
$ python exceptions.py intervals.txt
> /home/radorado/code/Talks/EuroPython2017/exceptions.py(26)parse_line()
25
---> 26 return parse_datetime(first), parse_datetime(second)
27
ipdb> first
'2017-06-25 19:27:51'
ipdb> second
' 2017-06-25 19:42:51'
ipdb> exit
Exiting Debugger.
def test_parse_line(self):
input = '(2017-06-25 19:27:51, 2017-06-25 19:42:51)'
expected = (datetime(year=2017, month=6, day=25,
hour=19, minute=27, second=51),
datetime(year=2017, month=6, day=25,
hour=19, minute=42, second=51))
result = parse_line(input)
self.assertEqual(expected, result)
def parse_line(value: str) -> Tuple[datetime, datetime]:
parts = value.split(', ')
first = parts[0]
second = parts[1]
# Remove ()
first = first[1:]
second = second[0:len(second) - 1]
return parse_datetime(first), parse_datetime(second)
..
----------------------------------------------------------------------
Ran 2 tests in 0.004s
OK
$ python exceptions.py intervals.txt
Traceback (most recent call last):
File "exceptions.py", line 69, in <module>
main()
File "exceptions.py", line 59, in main
result = run(fname)
File "exceptions.py", line 49, in run
data = [parse_line(value) for value in contents.split('\n')]
File "exceptions.py", line 49, in <listcomp>
data = [parse_line(value) for value in contents.split('\n')]
File "exceptions.py", line 17, in parse_line
second = parts[1]
IndexError: list index out of range
def parse_line(value: str) -> Tuple[datetime, datetime]:
parts = value.split(',')
import ipdb; ipdb.set_trace()
first = parts[0]
second = parts[1]
# Remove ()
first = first[1:]
second = second[0:len(second) - 1]
return parse_datetime(first), parse_datetime(second)
$ python exceptions.py intervals.txt
> /home/radorado/code/Talks/EuroPython2017/exceptions.py(18)parse_line()
16 import ipdb; ipdb.set_trace()
17
---> 18 first = parts[0]
19 second = parts[1]
20
ipdb> parts
['(2017-06-25 19:27:51', '2017-06-25 19:42:51)']
ipdb> cont
> /home/radorado/code/Talks/EuroPython2017/exceptions.py(18)parse_line()
16 import ipdb; ipdb.set_trace()
17
---> 18 first = parts[0]
19 second = parts[1]
20
ipdb> parts
['(2017-06-25 19:43:51', '2017-06-25 19:58:51)']
ipdb> cont
> /home/radorado/code/Talks/EuroPython2017/exceptions.py(18)parse_line()
16 import ipdb; ipdb.set_trace()
17
---> 18 first = parts[0]
19 second = parts[1]
20
ipdb> parts
['(2017-06-25 19:59:51', '2017-06-25 20:14:51)']
ipdb> cont
> /home/radorado/code/Talks/EuroPython2017/exceptions.py(18)parse_line()
16 import ipdb; ipdb.set_trace()
17
---> 18 first = parts[0]
19 second = parts[1]
20
ipdb>
def main() -> None:
fname = sys.argv[1]
from ipdb import launch_ipdb_on_exception
with launch_ipdb_on_exception():
result = run(fname)
if result:
print('There are overlapping intervals')
sys.exit(1)
print('All good.')
$ python exceptions.py intervals.txt
IndexError('list index out of range',)
> /home/radorado/code/Talks/EuroPython2017/exceptions.py(17)parse_line()
15
16 first = parts[0]
---> 17 second = parts[1]
18
19 # Remove ()
ipdb> parts
['']
def run(fname: str) -> bool:
with open(fname, 'r') as f:
contents = f.read()
data = [parse_line(value)
for value in contents.split('\n')
if value.strip() != '']
result = overlapping_intervals(data)
return result
Sometimes, the library or framework is going to tell you that you messed up.
ImproperlyConfigured, ValidationError, PermissionDenied, etc.
Fixes here should be easy. Just read the exception.
You run your code. There's no exception. There's no answer.
Level of frustration:
1
2
3
Fixing an error is much better than not knowing what to fix.
Commented everything in users/tasks.py -> ✓
Comment parts of code until everything's working
Level of frustration:
1
2
3
Everything's working but the answer is the wrong one.
I always look first for errors in the tests. A lot of times, this is the case.
some_object.refresh_from_db()
Is the input correct? Maybe you need to validate things.
This is often the case with implicit type castings.
Are you using external library? Have you implemented it yourself?
ipdb on every line & trace the value of everything. Understand where in the algo things start to get wrong.
Perhaps there are more dimensions to the problem.
For example - you are dealing with some kind of timelines but have implemented only the future & you need to support the past.
Starting fresh & going to the whiteboard always helps here.
You may be solving the wrong thing.
Understanding & learning more is the way to go here.
By Hack Bulgaria