Intro to Python

Goals

• The basics of Python scripting
• Where to find help on advanced topics
• Structuring your workflow
  • Wrangling Linux
  • Using online resources to maximize efficiency
  • How to go from idea to product
  • Structuring projects so they are re-usable
• Monkey-see, Monkey-do

$(whoami)

• Tech enthusiast (a.k.a a geek)
• Wannabe programmer (In a former life)
• Hobby Scripter
• Overall Lazy Person (Why this matters...?)
• History
  • Started Programming and Scripting in 2000
  • Worked as computer programmer before joining military (.NET, Python, etc.)

Course Breakdown

• Day 1 (Basics)
  • Environmental Familiarization
    • Command Line Bootcamp
    • Common software
  • Python Basics
    • How to write and run script
    • Keywords
    • Simple I/O
    • Intermediate Concepts (...maybe)

Course Breakdown (Cont.)

• Day 2 (Project Structure)
  • Using external resources
    • more git / github.com
  • Intermediate to Advanced Topics
  • How to structure projects internally

Course Breakdown (Cont.)

• Day 3 (Advanced Topics / Practicals)
  • Practical Exercises
  • Object Oriented Programming (OOP)

Module 1

"Conducting atmospherics"

Environment

• Linux
  • Arch (Antergos installer)
    • Pre-configured
  • Gnome 3
  • Why?
• Common Software
  • Guake
  • Gedit
  • Sublime Text
  • PyCharm
  • Haroopad
  • Meld

Before we start...

• F12 (Drop Down Terminal)
• Windows Key (App Search)
• Ctrl+Alt+{Up|Down} (Change Workspace)
• Important Directories
  • ~/Projects
  • ~/Desktop/RamDisk
  • /mnt/xfer -> ~/Desktop/Transfer
• Test:

student@pyclass02: ~
➜ ping -c 3 www.google.com

student@pyclass02: ~
➜ sudo pacman -Syu

Taking Notes

• I recommend using markdown (Haroopad is the editor)
• Markdown is a plain text syntax that allows code formating and highlighting and rendering
• Markdown in 30 seconds
   

# Title
## Sub Title
### etc. etc. etc.
    

```python
code goes here
```

Comments about `commands` 

You can also use **bold** , *emphasis* , 
++underline++ , ~~strikethrough~~ , ==Highlight==

Even [hyper-links](www.someurl.com)

Module 2

Linux / Terminal Bootcamp...yay

Why use the terminal?

• Speed
• Scriptability
• Flexibility
• Heavy text file use
• Dirty secret...

Terminal View

user@host: ~
➜ 

• Using zsh
• Accesss gauke via F12 (or opening a terminal program)
• 'user' is your username
• 'host' is the computer name of the computer you're logged into (ssh anyone?)
• Test

user@host: ~
➜ sudo -i

root@host: ~
➜  

Key Concepts

• Everything is a file (including directories)
  • Absolute paths start with a leading '/'
  • Relative paths start with anything else
• 'Hidden' files start with a leading '.'
• Many configurations and settings are simply text files
• Many GUI programs simply 'wrap' command line programs
• Commands can be 'linked' (or 'chained') with one another

Common Comamnds

• ls
• cd
• mkdir
• rm
• cat
• tail
• head
• echo
• cp
• mv

• touch
• sed
• awk
• xargs
• xd
• ln
• sudo
• su
• tree
• pwd

• type
• clear
• exit
• find
• grep
• man
• curl
• more
• less
• chmod
• and MANY more

Special Characters

• |
• >, >>
• <
• &
• &&, ;
• ||
• !!
• $
• #
• .
• ..
• ~

• Pipe
• Output, Append
• Input
• Background
• And (if successful), Next
• Or (If failed)
• Previous
• User
• Root
• Current Directory
• Parent Directory
• Home Directory

Environmental Variables

• Starts with a '$'
• Typically ALL_CAPS (No spaces, underscores instead)
• Can be saved via 'export'
• Typically placed in 'profile' file
  • /home/$USER/.zshrc (or .bashrc)
• Common preset variables (for full list use 'env')
  • $HOME
  • $USER
  • $PATH
  • $SHELL

Special Files / Directories

• /home/$USER
• /dev
  • /dev/null
  • /dev/zero
  • /dev/urandom and random
  • /dev/sda (or b, or c, etc.)
• /etc
• 'bin' directories
  • Specifically those listed in $PATH

    • $ echo "${PATH//:/\n}"

Idioms

user@host: ~
➜ fake -s

user@host: ~
➜ fake --something

user@host: ~
➜ fake -s option

Basic Usage

Switches (Options and Flags)

user@host: ~
➜ fake 

user@host: ~
➜ fake | other

Piping (Chaining/Combining)

user@host: ~
➜ fake > somefile.txt

Redirecting

Advanced Idioms

user@host: ~
➜ fake >/dev/null 2>&1

Suppress Errors

Suppress Errors and Output

user@host: ~
➜ fake 2>/dev/null

user@host: ~
➜ fake << EOF
Type the input for the command
EOF

Type Input

user@host: ~
➜ fake > somefile.txt

Redirecting

Getting Help

• Almost no one remembers all options and arguments
• Getting help, try

user@host: ~
➜ fake -h

user@host: ~
➜ fake --help

user@host: ~
➜ man fake

Google is a great option too!

• In most documentation [] means optional, <> means required

Making a Cheat-sheet

• First use trial and error to fine tune commands and pipes
• Make a cheat-sheet of useful commands
  • Save in ~/.dotfiles/scripts/usefulcommand.md
  • Use the same format throughout the cheat-sheet

## awesome command for stuff
```bash
super -d | fake --command | yay
```

• Then search with howto

user@host: ~
➜ howto 'awesome command'

user@host: ~
➜ howto stuff

Custom Commands

• Alias commands to make them easier to remember
• Example, in archlinux to update all packages use the command 'sudo pacman -Syu', perhaps it's easier to remember 'update' ?
• Aliasing is simply match and replace
  • whatis
  • type
• Consider security (to sudo or not sudo?)

user@host: ~/Desktop
➜ echo 'alias update="sudo pacman -Syu"' >> ~/.zshrc

Pop Quiz!

Module 3

Why Python? ...sell me this pen

What is Python?

• A scripting language (specifically an interpreter)
• Uses text files (typically ending with .py) to perform specific tasks
• Each text file is read and executed from top to bottom
  • NOT compiled  
  • ...Buuuuut JIT compiling does happen (.pyc)
• Must have interpreter installed to run python scripts

Why use Python?

• Tasks that are: Repeatable, Mundane, Complex, Precise, Time Consuming, etc.
• Bridge the end-user technology gap
• Rapid prototyping of larger projects
• Brevity
• Short development time
• HUGE repository of libraries and modules (i.e. VERY popular)

Are there any downsides?

• Execution speed (vs. development speed)
• Current version split (2.x vs 3.x)
• Initial learning curve
• Over-engineering
• Age (technological advances in parallelism)
  • Competitors: Go, Rust, Ruby, Julia, Bash, etc.

Module 4

Are we there yet?

Prep Phase (First Test)

• Create a new folder inside the Projects folder named hello

• Create a new file named 'hello.py'
• Open the file and start scripting...

Preparation Solution

user@host: ~
➜ cd Projects

user@host: ~/Projects
➜ mkdir hello

user@host: ~/Projects
➜ cd hello

user@host: ~/Projects/hello
➜ touch hello.py

user@host: ~/Projects/hello
➜ vim hello.py

Obligatory 'Hello World'

Solution

#!/usr/bin/env python
print('Hello world!')

Note: env python? Good idea, or no?

Execution Phase

• Optionally make script executable
• Run the script
• Observe the output
• Refactor
• Repeat

Execution Solution

user@host: ~/Projects/hello
➜ chmod +x hello.py

user@host: ~/Projects/hello
➜ ./hello.py
Hello World!

user@host: ~/Projects/hello
➜ python hello.py
Hello World!

Module 5

The Basics...finally!

Statements

• Each line in a script that performs an operation is called a statment
• Previously we had one statement
• Statements must be separated by a return (enter key)
  • Linux vs Windows 'return'

Statements Exercise

• Create a new script named 'statements.py'
• On three lines, the script should print
  • Your first name
  • Your last name
  • Your age

Example Output

user@host: ~/Projects/statements
➜ ./statements.py
Kevin
Knapp
29

Solution

print('Kevin')
print('Knapp')
print(29)

Variables

• 'Buckets' of information (vs. literals)
• Can hold many different types of information
• Must start with a letter
• Typically snake_case_names
• Use descriptive names when possible
• Cannot be a keyword
• Assigned and re-assigned using '='

variable_name = <literal> OR/AND <other variables>

Example

name = 'something'
age = 29
more_text = a_var

Variable Exercise

• Create a new script called variables.py
• On three lines print your first name, last name, and age using only two variables and without printing literals

Variable Solution

name = 'Kevin'
print(name)
name = 'Knapp'
print(name)
age = 29
print(age)

Types

• Dynamic Types
• Common Types
  • strings
  • numbers (int, float, etc.)
  • characters
  • bools
  • lists, dicts, sets, and tuples
  • objects
  • None

Math

• Common math operators
  • +, -, *, /, %, **
  • +=, -=, /=, %=
• Boolean math operators
  • ==, !=, not (!), or, and
  • >, <, <=, >=
• Pay attention to types!

Math Exercise

• Create a new script called math.py
• add two numbers, divide two numbers, multiply two numbers, and subtract two numbers, and modulating two numbers, displaying all results
• Add two strings and display the results
• Add an integer and float, display the results
• Add two floats and display the results

Math Solution

num1 = 2
num2 = 2
both = num1 + num2
print('Adding: ', both)
both = num1 - num2
print('Subtracting: ', both)
both = num1 / num2
print('Dividing: ', both)
both = num1 * num2
print('Multiplying: ', both)
both = num1 % num2
print('Modding: ', both)
text1 = '2'
text2 = '2'
both = text1 + text2
print('String and String: ', both)
fl1 = 2.2
int1 = 2
both = fl1 + int1
print('Float and Int: ', both)
fl2 = 2.2
both = fl1 + fl2
print('Float and Float: ', both)

Deeper into Types

• Why did adding two strings simply concatenate them?
  • Operator Overloading
• Why can 2.2 + 1 = 3?
  • Implicit Casting
  • Computers use base 2 numbers, instead of base 10 that humans use
• Strings are special...

Strings

• In Python 3.x all strings are a sequence of valid UTF-8 Unicode characters
• ASCII vs UTF-8
  • 1 byte vs maybe several
  • ASCII byte strings in Python 3 (str type, or b'literal')
• Hidden characters in strings
  • Escape character is \ (i.e. 'string with\'quotes\'')
• Common hidden characters
  • \n
  • \t
  • \c\r
• Strings are simply lists of characters and can be sliced (more on that later...)

Lists

• An unordered sequence of values
• Values do not need to be same type
• Formed using the [ and ] characters, separating values by , character
• Can be indexed via an integer
  • Starts at 0!!
  • A negative integer means, "From the end"

List Excercise

• Create a new script named lists.py
• Create a list variable that holds your first name, last name, and age.
• Display all the values of the list

List Solution

my_list = ['Kevin', 'Knapp', 29]
print(my_list[0])
print(my_list[1])
print(my_list[2])

List Slicing

• Lists can be sliced to refer to a specific section of the list using the [:] characters and 0, 1, or 2 numbers
• Using 0 numbers refers the to whole list

my_list[:]

• The number before the : is the start index
• The number after the : is the end index
• Omitting either says, "Everything till the end, or from the beginning" respectivly

from_start_to_index_5[:5]
from_index_2_to_end[2:]
from_index_1_to_4[1:4]

• A negative number for either start or end, simply reverses the index (i.e. from start or from end)
• Start must ALWAYS be below End

Comments

• Comments are not evaluated statements
• Start with either # or triple ''' (If starting with triple ''' they MUST also end with ''')
• It is a VERY good idea to comment your code heavily
• Good style is to start each file with a long comment covering how to use the file, what it does, versions, and license information.

Console I/O

• We've already been using the O
• Console Input reads a string from the user (including hidden characters such as new lines (\n) ) and stores it in a variable
• When reading numbers from a user, they must be converted from a string into a number before use

Guessing Game

• To demonstrate the following principals we will create a guessing game
• The user must enter his name and will try to guess a randomly selected number known only to the computer
• After each guess his score will be incremented by 1
• The goal is to have the lowest score when the number is guessed
• After each guess the computer will tell the user if the guess was either too high or two low

guess v1

• Start by creating a new script called guess.py
• Use a variable to store the secret number
• Ask the user his name
• Use a variable for the score
• Let the user make a guess
• For now simply display the guess and exit

guess v1

#!/usr/bin/env python
# Guess v1
# Simply guessing game for learning Python
# Licensed with MIT
#
secret_number = 34
name = input('Enter your name: ')
guess = input('Enter your guess: ')
print('{} guessed {} but the number was {}'.format(name, guess, secret_number))

Actions (aka Method Calls)

• Many variables can be 'acted upon' i.e. have various actions performed on them. These are known as method calls.
• The usage is <variable>.<method> where variable would have the action <method> applied to it.
• Methods are type specific and will be covered later (i.e. number variables don't have the same actions as string variables).
• Arguments can be based to the methods inside ( and ) characters. The method uses these to perform some action.
• Some literals, such as strings can use method calls as well i.e. the 'string'.format() method call. This replaces all occurances of {} with the arguments passed to the "format" action.

Using User Input

• As stated earlier, in order to use input such as numbers they must first be converted from strings to numbers. To demonstrate this do the following

#!/usr/bin/env python
string_num = input('Enter a number: ')
real_num = 2
result = string_num + real_num
print(result)

• If you enter 2 at the prompt the result will be 22 because the string literal '2' is not the same as integer literal 2. Thus with implicit casting, the integer literal is cast to a string and they are concatenated.

Parsing Numbers in Strings

• In order to parse numbers inside strings, i.e. convert them you must use explicit casting
• To explicitly cast, wrap the variable or expression in () following the type you wish to convert to

str_age = input('Enter your age: ')
future = int(str_age) + 2
print('In 2 years you will be {} years old'.format(future))

Not all types can be cast to each other!!

Conditionals

• Sometimes you want to take different actions based on certain conditions
• Python provides if/else statements
• Conditions use Boolean logic to determine which path to take

if some_expression_or_variable:
    # Actions take if TRUE
else:
    # Actions to take if FALSE

• Python also provides multi-condition statements by adding elif

if some_expression_or_variable:
    # Actions take if TRUE
elif some_new_test:
    # Actions to take if TRUE
else:
    # Actions to take if all tests are FALSE

Conditionals Cont.

• if/elif/else statements can be nested inside each other to form complex logic paths
• if statements do not need to be followed by an elif or else clause
• elif statements do not need to be followed by an else clause
• Try to avoid nesting too deep (usually greater than 3 levels is too much)

Conditional (Boolean) Logic

• Common operations are: not, and, or, ==, !=, <, >, <=, >=
• Operations can be compounded

guess v2

• Now that we can convert strings to numbers and compare, lets update the guessing game
• After getting the users guess, compare it to the secret number and inform the user if he is too high, too low, or spot on

guess v2

#!/usr/bin/env python
# Guess v2
# Simply guessing game for learning Python
# Licensed with MIT
#
secret_num = 34
name = input('Enter your name: ')
str_guess = input('Enter your guess: ')
num_guess = int(str_guess)
if num_guess == secret_num:
    print('{}, you guessed it!'.format(name))
elif num_guess < secret_num:
    print('{}, your guess is too low'.format(name))
else:
    print('{}, your guess is too high'.format(name))

Assumptions (no >)?

Loops

• Sometimes you need to do a specific operation multiple times
• Python provides loops for these occasions
• There are two types of loops in Python
  • for
  • while

for Loop

• Used when you need to do something a specific number of times, or for an entire collection of values
• The syntax is

for value in some_collection:
    # do something

• For example, to print every item in a list

my_list = ['item 1', 'item 2', 'item 3', 'item 4']
for item in my_list:
    print(item)

for Loop Cont.

• If you want to do something a specific number of times you can use the range() function (Functions will be covered later)

# say 'Hello' 10 times
for _ in range(10):
    print('Hello')

• Notice we used the '_' character, that's because we didn't need the 'item' variable (in this case it would be a number). Alternatively if we wanted to use it to count we could do

# Count to 10
for num in range(1,11):
    print(num)
    print(',')

Note the start and end values passed to range

while Loop

• The while loop is like an if/elif/else statement as it continues to loop while a particular variable or statement is True, and stops when the variable or statement is false. For example, to count to 10

i = 1
while i < 11:
    print(i)
    print(',')
    i = i + 1

Special Loop Conditions

• There are times you will want to end a loop early, or skip iterations of a loop. Python provides break and continue statements for these.
• continue instantly jumps back to the beginning of the loop and continues where it left off

# only print odd numbers
for i in range(10):
    if (i % 2) == 0:
        continue
    print(i)

• Break stops the loop immediately and goes to the next statement AFTER the loop and is helpful with infinite loops

# infinite loop stop after printing 5 times
while True:
    counter = 1
    print('Hello')
    counter += 1
    if counter == 5:
        break

guess v3

• Update the guessing game to continuously ask for guesses until the user gets the correct number, or presses 'q' for quit.
• If the correct score is guess, ask if a new player would like to try, and compare scores.
• Note, there are a number of ways to accomplish these tasks.

guess v3

#!/usr/bin/env python
# Guess v3
# Simply guessing game for learning Python
# Licensed with MIT
#
secret_num = 34
players = dict()
curr_player = input('Enter your name: ')
players[curr_player] = 0
while True:
    str_guess = input('Enter your guess[q to quit]: ')
    players[curr_player] += 1
    if str_guess == 'q':
        print('Results:')
        for p in players:
            print('\t{}\t{}'.format(p, players[p]))
        break
    num_guess = int(str_guess)
    if num_guess == secret_num:
        print('{}, you guessed it with a score of {}!'.format(curr_player, \
                                                        players[curr_player]))
        new_game = input('Would you like to play again?[y/n]: ')
        if new_game == 'y':
            name = input('New player name[enter for same player]: ')
            if name != '':
                players[name] = 0
                curr_player = name
            else:
                players[curr_player] = 0
            secret_num = 72
        else:
            print('Results:')
            for p in players:
                print('\t{}\t{}'.format(p, players[p]))
            break
    elif num_guess < secret_num:
        print('{}, your guess is too low'.format(curr_player))
    else:
        print('{}, your guess is too high'.format(curr_player))

Functions

• While the new guessing game works OK there are some flaws (ignoring error handling)
• First, we only have two secret numbers
• Second, we are repeating code in several places
• Luckily there are ways to de-duplicate code known as functions and modules
• Functions and modules are (usually) small sections of code that are essentially copy and pasted automatically for you
• This also means if you make a change to the way a function or module works, you only need to change it once!

Functions

• Functions are small re-usable sections of code
• They may be called (i.e. used) anywhere and work similar to method calls
• Like method calls, you can pass additional arguments via variables and literals to the functions in order to do some work or processing
• Unlike method calls, they do not act on a specific variable or literal EXCEPT those passed into them
• Functions also have an optional RETURN variable, which can be thought of as the results of the processing or work

Function Syntax

• Functions start with the def keyword, then the name, then parenthesis containing any optional arguments
• Arguments are positional and mandatory (with one exception, more on this later...)
• If functions wish to send the results back, they use the return keyword followed by the results variable or literal

# A function that takes 2 arguments and doesn't
# return a result
def my_function(some_arg, other_arg):
    # do some work here...

# A function that takes 0 arguments and
# returns a result
def my_other_function():
    # do some more work...
    return True

Function Use

• Functions are used by typing the name, and supplying any arguments they are expecting
• If there is a returned result, you save that in a variable with the = sign, or ignore it (not always a good idea)

a = 1
b = "two"
my_function(a, b)

c = my_other_function()

Note: functions can use literals too!

Modules

• Like functions, modules are re-usable code sections.
• Unlike functions, modules are an entire file and include many different functions and variables
• Modules are used to group like code together for organization
• Python includes MANY pre-made modules that include all kinds of useful functions and variables (everything from math, to networking)
• In order to use functions and variables contained in a module, you must import them into your scripts

Module Example

• As a simple example imagine we had a module named animals.py which contained two functions and one variable for use by other programs:

#!/usr/bin/env python
# animals.py
FAVORITE_ANIMAL = 'eagle'

def make_cats_bark():
    print('The cat says, woooof!')

def make_cows_tweet():
    print('The cow goes tweet, tweet!')

• We can then import this module and use the functions and variables it contains by accessing them via the . character

#!/usr/bin/env python
# madscience.py
import animals

print('What is my favorite animal? {}'.format(animals.FAVORITE_ANIMAL))
print('Science is nuts...')
animals.make_cows_tweet()
animals.make_cats_bark()

guess v4

• Lets use our new knowledge of functions and modules to clean up our guessing game
• Since Python provides a way to generate random numbers, we will use that module to get a new number each time
• Also, clean up the duplication of setup code (making names, scores, random numbers, etc.)

guess v4

#!/usr/bin/env python
# Guess v4
# Simply guessing game for learning Python
# Licensed with MIT
#
import random

secret_num = 0
players = dict()
curr_player = ''

def new_secret():
    return random.randint(0,100)

def setup(n):
    global players    
    global secret_num
    global curr_player
    if n:
        curr_player = n
    players[curr_player] = 0
    secret_num = new_secret()

def end():
    print('Results:')
    for p in players:
        print('\t{}\t{}'.format(p, players[p]))

setup(input('Enter your name: '))
while True:
    str_guess = input('Enter your guess[q to quit]: ')
    if str_guess == 'q':
        end()
        break
    num_guess = int(str_guess)
    players[curr_player] += 1
    if num_guess == secret_num:
        print('{}, you guessed it with a score of {}!'.format(curr_player, \
                                                        players[curr_player]))
        new_game = input('Would you like to play again?[y/n]: ')
        if new_game == 'y':
            setup(input('Enter your name[enter for same player]: '))
        else:
            end()
            break
    elif num_guess < secret_num:
        print('{}, your guess is too low'.format(players[curr_player]))
    else:
        print('{}, your guess is too high'.format(players[curr_player]))

Testing with a REPL

• Read Evaluate Print Loop
• Python's REPL is called IDLE
• Accessed via 'python' command
  • To stop, type 'quit()'
• Most python modules are available, plus some extras (namely dir() )
• Sessions non-persistent
• Great for testing (and short term memory loss, aka 'getting old')

Summary

• This only scratches the surface
• Variables are buckets or placeholders of information
• There are various types of data, such as numbers, strings, lists, etc.
• You can control the flow of execution via if/elif/else statements
• You can repeat tasks via for and while loops
• You can re-use code with functions and modules
• Use IDLE to test ideas and remember how things work

Module 6

Project Structure..or lack there of

Split Projects Up

• Use a 'main' project file with project name
• Take like functionality and move it into other files (modules)
• Create helper modules with like functions
• Separate display code from 'business logic'
  • Akin to 'MVC' model
• Use packages...

Typical Project Structure

user@host: ~/Projects 
➜ tree fake
fake
├── examples
│   └── fake.py
├── fake
│   ├── fake.py
│   ├── somemod.py
│   └── somepkg
│       ├── __init__.py
│       └── someothermod.py
├── LICENSE
├── README.md
└── tests
    └── fake_tests.py

Packages

• Any folder with a blank __init__.py file becomes a package
• __init__.py files can contain real code
• Make packages of like files and functionality to make imports logical

Make Libraries

• Prefer libraries instead of top down scripts

import sys

def awesome_function():
    # do stuff

def main():
    # Start you script here!

if __name__ == '__main__':
    sys.exit(main())
    

• This allows you use functions from your files without running the file, it also allows you test your libraries
• Also becomes good shell stuard by maintaining exit codes (all functions implicitly return None)
  • Can add args to main if needed

Where to Start?

• List Requirements
  • Prioritize Functionality (Need vs Want)
  • If creating a library, it may help to USE a mockup
• Divide functionality into modules
  • Data flow diagrams help
• Keep functions single task
  • preferably single screen
  • Fail early (i.e. test for failures, assume success)
• DOCUMENT
  • Document WHAT a function does, not HOW
• Start with core functionality
• Work outwards twoards UI
• UI should have no function with business logic

Module 7

Version Control...git it?

git

• Version control software that tracks changes to files
• Allows you (or multiple people) to contribute to a single project
• Allows multiple 'branches' to be worked on at the same time and merged together
• git is SUPER complex, we will only scratch the surface

git Basics

• Enable git tracking on a folder (i.e. project)

user@host: ~/Projects/Awesomeness
➜ git init

• By default you are working on the 'master' branch
• By default git tracks all files in the folder (if you add new files later use the command git add <files>)
• Work on files as normal
• Once you are ready to "save" your code issue the command

user@host: ~/Projects/Awesomeness
➜ git commit -a                       [git: master]

• This "saves" the files and asks you to add a message about what you changed
• Use detailed messages!
• To add the message inline use 'git commit -am "message"'

Branching

• Lets say you have a working script, but want to add a new feature or work on a bug, but you don't want to break existing code
• You can add a new branch
• This in effect copies your project, and allows you work on both the working code and the 'copy' code at the same time
• To branch, use

user@host: ~
➜ git checkout -b "branch name"

Branches (cont.)

• You switch back and forth between branches at will

user@host: ~
➜ git checkout awesomefeature       [git: master]

• Once you have a working feature, or bug fix you can 'merge' the two branches back into one

user@host: ~
➜ git checkout master        [git: awesomefeature]

user@host: ~
➜ git merge awesomefeature              [git: master]

• When you have no need for a branch anymore you can delete it

user@host: ~
➜ git checkout -d awesomefeature      [git: master]

github.com

• Online git repositories
• Allows you to save projects and work with others easily
• Free service requires public repositories (i.e. projects)
• Can be used for more than just scripting/coding

github.com Example

• Create github account
• Create  repo called guess-py
• 'clone' the repo onto your local machine
• copy the latest version of the guessing game into this repo
• Remeber to 'git add' the files
• Commit the changes
• Save the files to the remote repository

github Solution

user@host: ~/Projects
➜ cd Projects

user@host: ~
➜ git clone https://github.com/kbknapp/guess-py

user@host: ~/Projects
➜ cd guess-py

user@host: ~/Projects/guess-py
➜ cp -r ../guess/* .                   [git: master]

user@host: ~/Projects/guess-py
➜ git add .                          X [git: master]

user@host: ~/Projects/guess-py
➜ git commit -am "Initial commit"    X [git: master]

user@host: ~/Projects/guess-py
➜ git push origin master               [git: master]

Updating Remote Repos

• Say you or someone else updated your guess-py repo on another machine
• To get the latest and greatest

user@host: ~/Projects/guess-py
➜ git pull origin master                       [git: master]

Module 8

Intermediate Topics..."Slow down professor!"

Collections

• Ways to represent sets of items (some have been lightly covered)
• Common collections include lists, sets, dicts, tuples, and deques

List

• Unordered list of non homogenous items referenced by 0 based index
• Literal uses [ and ]
• Can be multi-deminemsional (i.e. a list of lists)
• Note: Special 'list comprehensions' are amazing

my_list = ['kevin', 'knapp', 28]

last_name = my_list[1]

# Sample multi-deminsional (a 3x3 grid)
multi_list = [ [1,2,3], [4,5,6], [7,8,9] ]

row_2_col_2 = multi_list[1][1]

# Sample comprehension
one_to_ten = [i for i in range(10)]

dict

• A.k.a. hash-map or a key-value list
• Keys do not need to be same type as value
• Literal uses {, :, and }
• Keys MUST not conflict
• Values accesses via [ and ] with the key

person = {'first': 'Kevin',
          'last': 'Knapp',
          'age': 29 }
my_age = person['age']

Set

• An unordered list of UNIQUE items
• Literals use { and }
• Cannot be access via [ and ], but can be iterated
• A good use is to remove duplicates from other collections

my_set = set()
my_set.add('kevin')

Tuples

• Multiple values pretending to be a single value
• Cannot be changed once created (immutable)
• Great for returning an immutable list from a function
• Literal uses ( and )
• Can use destructuring! (use _ to 'throw away' variables)
• Can be accessed via [ and ] with a 0 based index

my_tuple = ('kevin', 'knapp', 29)

_, last, _ = my_tuple

first = my_tuple[0]

deque

• A FILO stack (but can be used as a FIFO pipe)
• Must import collections (i.e. not just builtin per se)
• Can be accessed via [ and ] with a 0 based index

import collections
# Used as FILO stack
filo = collections.deque()
filo.append('kevin')
filo.append('knapp')
print(filo.pop())
print(filo.pop())

# Used as FIFO pipe
fifo = collections.deque()
fifo.appendleft('kevin')
fifo.appendleft('knapp')
print(fifo.pop())
print(fifo.pop())

Keyword Arguments

• Keyword Arguments (a.k.a. kwargs/kargs) allow you to have optional arguments to a function or method (along with mixing the order)

def cool_function(name='kevin', age=29):
    # do stuff here
    return (name, age)

# use the function
n, a = cool_function(age=50)
print('name={}, age={}'.format(n,a))

• kwargs can be mixed with positional args too

More on Imports

• Better practice to import only what you need
• Also allows you specify ONLY the function or variable you need and not the whole module
• For our previous deque example

from collections import deque

filo = deque()

• The lazy way (more chances of namespace collisions)

from collections import *

filo = deque()

• Alias imports (imagine you have two libraries with similar functions...or just really long names)

from collections import deque as filo

my_stack = filo()

File I/O

• Being able to read and write to files is fundamental
• You can either read and write files by line or by byte (depending on your needs)
• All files must first opened, written or read from, then closed (easy to use 'with..as' keywords)
• Files may be opened in read only, write, or append modes

Simple File I/O

• Open a file and read all lines
• Output each line's length to a new file
• Then the total lines, and average length

Sample Output:

line 01 - 234

line 02 -   28

2 Lines total (126 Avg. Len)

File I/O Example

t_lines = 0
t_len = 0

lines = []

with open("line_count.txt","r") as f:
    for line_no, line in enumerate(f):
        t_lines += 1
        l_len = len(line.strip())
        lines.append([line_no, l_len])

with open("line_output.txt", "w") as o:
    for l in lines:
        t_len += l[1]
        o.write('Line {} - {}\n'.format(l[0], l[1]))
    o.write('Total lines {} - Avg Len {}'.format(t_lines, int(t_len/t_lines)))

Error Handling

• Some errors you can deal with programatically
• Python allows you handle, or silence errors (not always a good idea) with the 'try/except/else/finally' keywords

try:
    # do something that might fail
except:
    # do something if it failed
else:
    # do something ONLY if it DIDN'T fail
finally:
    # do something no matter what (including fails)

Note: All keywords do not need to be used

Handle Specific Errors

• Sometimes you only want to handle specific errors
• Python allows you to do this with builtin and custom exceptions

try:
    answer = 2 / 0
except ZeroDivisionError:
    print('Divided by zero')
except:
    print('Something else happened')

print('But still kicking!')

Custom Exceptions

• If the builtin exceptions aren't enough, you can build your own
• You can then 'cause the error' with the 'raise' keyword
• Keep in mind, exceptions are expensive

class LowIQException(Exception):
    pass

def get_name():
    name = input('Enter your name: ')
    if not name:
        raise LowIQException
    return name

user = ''
try:
    user = get_name()
except LowIQException:
    user = input('Ok dummy...PLEASE enter your name: ')

print('{} joined the discussion'.format(user))

Module 9

Get friendly with these imports...

Common Modules

• os (and os.path)
  • OS specific features such as working with directory and file names, environmental variables, etc.
• sys
  • Handles system level tasks like proper exits
  • Allows simple external program calls
  • Basic manipulation of program arguments
• subprocess
  • More advanced, and powerful handling of spawning processes and communicating between external programs
• itertools
  • Amazing gems for handling data that needs to be iterated over

Module 10

Advanced-ish... :|

First Class Citizens

• Combining what we learned earlier we can create some powerfully dynamic structures
• In order to do so, we need to use functions as fist class citizens, meaning you can assign functions to variables and pass them as arguments to other functions

First Class Citizen Eample

• Passing functions as arguments or variables is easy

def my_super_function():
    print('inside my super function!')

def my_lame_function(func):
    func()
    print('Someone is using my lame function...')

my_lame_function(my_super_function)

Lambdas

• Lambdas are single expression functions assigned to a variable
• Lambdas take 0 or more arguments and perform ONLY a single expression

dbl = lambda x: x*x

answer = dbl(2)

print(answer) # prints '4'

List Comprehensions

• An AWESOME feature
• Allows one to generate a list on the fly using a literal [ and ] with a for loop
• Can be used anywhere a list is epxected

three_by_three = [[0 for _ in range(3)] for _ in range(3)]
# [ [0, 0, 0], 
#   [0, 0, 0],
#   [0, 0, 0]
# ]

• Can be used to make a list of lists easily

one_to_ten = [i for i in range(10)]

Random MAC Address

• An example showing use of lambdas and list comprehensions to make a random MAC address

from random import randint

hex_byte = lambda: '{:02X}'.format(randint(0,255))
rand_mac = lambda sep: sep.join(hex_byte() for _ in range(6))

colon_macs = [rand_mac(':') for _ in range(3)]
hyphen_macs = [rand_mac('-') for _ in range(3)]

print(colon_macs)
print(hyphen_macs)

Objects (OOP...not oops)

• If lists, sets, dicts, etc. aren't enough you can make your own types
• Each type is called a object
• Objects have various fields (variables) and functions (methods) associated with them
• For an example, we will make an Animal object

Classes

• Classes are the basic building block of an object
• They are the wrapper for everything else
• The basic structure is

class MyClass(object):
    # Stuff goes here

• This definition serves as the blue print for the object

Fields

• Variables contained within a class are called the fields
• A class can have as many fields as required or none
• Fields are accessed in the blueprint via the keyword 'self' and the dot operator
• Since we are writing the blueprint for the object, we need a way to refer to brick-and-mortar object which is 'self'
• This also eliminates the need for 'global' (A good thing!)

class Animal(object):
    legs = 4

dog = Animal()
print('The dog has {} legs.'.format( dog.legs ))

Methods

• Methods are functions that work on objects
• Methods ALWAYS have their first arguement be an invisible 'self' which gives the blueprint access to the class

class Animal(object):
    legs = 4
    
    def print_legs(self):
        print('This animal has {} legs.'.format(self.legs))

spider = Animal()
spider.legs = 8

spider.print_legs()

Constructors

• Constructors are used to make new objects, typically with default values
• A special method of __init__(self)

class Animal(object):
    def __init__(self, num_legs):
        self.legs = num_legs

    def walk(self):
        print('This animal walks with {} legs.'.format(self.legs))

cat = Animal(4)
kangaroo = Animal(2)

cat.walk()
kangaroo.walk()

animal.py

# animal.py
class Animal(object):
    def __init__(self, kind='animal', legs=0, sound=''):
        self.num_legs = legs
        self.makes_sound = sound
        self.kind = kind
    
    def speak(self):
        if self.makes_sound:
            print('The {} goes {}'.format(self.kind, self.makes_sound))
        else:
            print('The {} is silent.'.format(self.kind))

cat = Animal(kind='cat', legs=4, sound='meow')
dog = Animal(kind='dog', legs=4, sound='woof')
spider = Animal(kind='spider', legs=8)

cat.speak()
dog.speak()
spider.speak()

Privacy

• All variables within a class are inside the global space so there is a possibility of naming conflicts
• If you want a variable or method to be 'private' meaning not for others to use except you it's good practise to name the variable or function starting with a single _ such as _my_func()
  • This does not actually make it private, it just means, "Please don't use this method, variable, or function directly please."
• If you start a function, method, or variable with two _'s such as __my_private_func() Python will actually mangle the name, meaning it's very hard to confict with (but still not really private!!)

Inheritance

• Objects and classes can actually inherit from each other in a tree like fashion, let's add to animal.py to add a dog and monkey class and see why it maters

Inheritance

# animal.py
class Animal(object):
    def __init__(self, kind='animal', legs=0, sound=''):
        self.num_legs = legs
        self.makes_sound = sound
        self.kind = kind
    
    def speak(self):
        if self.makes_sound:
            print('The {} goes {}'.format(self.kind, self.makes_sound))
        else:
            print('The {} is silent.'.format(self.kind))

class Dog(Animal):
    def __init__(self, name):
        Animal.__init__(self, kind='Dog',legs=4,sound='Woof')
        self.name = name

    def attack(self, who):
        print('{} is attacking {}!'.format(self.name, who))
    

class Monkey(Animal):
    def __init__(self):
        Animal.__init__(self, kind='Monkey',legs=2,sound='Oooo Ooo OOooh!')

    def climb(self):
        print('This monkey climbs like a tarzan!')


sparky = Dog('Sparky')
george = Monkey()
spider = Animal(kind='Spider',legs=8)

sparky.speak()
george.speak()
spider.speak()

sparky.attack('Luis')

george.climb()

...More (Outside the Scope)

• Closures
• Decorators
• Abstract Classes
• Meta-programming
• Threading
• Coroutines
• ...etc.

Practical Application 1

Duck, Duck, Duck...Goose!

Requirements

• Calculate every permutation given a set of constraints
• Constraints may be provided by the user
  • Upercase, lowercase, special chars (and which), numbers
  • Number permutations to calculate
  • Hash algorithm to use (if any)
• Display a percentage and calculated size prior to execution
• Permutations may be with or without hash
• Save results to a file
  • Bonus if file is split per size requirements

Find

9970d8254eacdf09e8e6097a12874728

Practical Application 2

Eyes in the back of my head!

Requirements

• Given a CSV file with multiple fields, such as a log file
• Find all occurance where a field either
  • matches a given set
  • occours multiple times over a span of another given field
• List all results in priority order