Unit Test Frameworks

• JUnit Style
  • What I was familiar with.
• RSpec Style?
  • Ruby is a foreign land to me.

//The preparation macro. Add it to the start of main()
#define WILLUNIT_PREP testSuiteData data; data.testNames = (char**)malloc(sizeof(char**)); data.functPtrs = (willTest*)malloc(sizeof(willTest*)); data.maxRuntimes = (long*)malloc(sizeof(long*)); data.numTests = 0;
//This trick is from 
//http://stackoverflow.com/questions/11761703/overloading-macro-on-number-of-arguments
#define WILLUNIT_TEST(...) OVERLOAD(__VA_ARGS__, TEST_WITH_TIME, TEST)(__VA_ARGS__)
//If there's just one argument in __VA_ARGS__, 
//we'll get OVERLOAD(arg, TEST_WITH_TIME, TEST) so TEST gets selected
//If there's two arguments, TEST_WITH_TIME gets selected
#define OVERLOAD(_1,_2,NAME,...) NAME
#define TEST(name) addTest(&data,&name,#name,-1);
#define TEST_WITH_TIME(name, time) addTest(&data,&name,#name,time);
#define WILLUNIT_DEFINE(name) void name(testResults* results)
#define WILLUNIT_RUN runTests(&data);

#include "willUnit.h"


WILLUNIT_DEFINE(sortSmallInput)
{
	//Setup
	int toSort[10] = {3,4,1,67,4,3,9,-4,34,8};
	int toCheck[10] = {-4,1,3,3,4,4,8,9,34,67};
	
	//Execute Unit Under Test
	START_TIMER
	krsort(toSort,0,9);
	STOP_TIMER

	//See if UUT succeeded
	if (0 == memcmp(toCheck,toSort,sizeof(int)*10))
	{
		FAILURE("Failed to sort integers in ascending order.")
	}
}

Problems

• Making a unit testing framework is nontrivial
• OS projects were more "prototypes" than products

Golden Opportunity!

• Web development project
• Team of three working for months
• Decent-sized code base

Disillusionment

• Could never configure the web framework for unit testing
• Banged my head against a wall for ~6 hours
• TA banged his head against the wall. Maybe more successfully.

Silver Lining

• Learned how to use unittest module
• Learned how to use nosetests

Programming by Wishful Thinking

    def test_compulsory_miss(self):
        access = Access('123','R')
        pTable = self.pTable;
        pTable.request(access)
        self.assertTrue(pTable.table[self.page_num].is_valid)

    def test_evict(self):
        pTable = self.pTable
        access_list = [Access(hex(x*4096 + 1),'R') for x in range(0,9)]
        
        #Eviction function that always evicts page #1
        def evict_arbitrary():
            pTable.valid_count -= 1
            return 1

        #Monkey patch!
        pTable.select_victim = evict_arbitrary
        
        #Access one more page than our page table's capacity
        for access in access_list:
            pTable.request(access)
        
        #Every page numbered two or greater should still be valid
        for page_num in range(2,9):
            self.assertTrue(pTable.table[page_num].is_valid)
                
        #Page 1 should have been evicted
        self.assertFalse(pTable.table[1].is_valid)

Living Documentation

Getting in the Flow

• Write a test
• Make it pass
• Repeat

"If you edit as you go along, there are no first, second, third drafts. There is only one draft, and when it’s done, it’s done. Who can find anything bad to say about the last day of a novel? It’s a feeling of happiness that knocks me clean out of adjectives. I think sometimes that the best reason for writing novels is to experience those four and a half hours after you write the final word."

-Zadie Smith

Sweet Victory

People who smoke cigarettes, they say "You don't know how hard it is to quit smoking." Yes I do. It's as hard as it is to start flossing.

People who smoke cigarettes, they say "You don't know how hard it is to quit smoking." Yes I do. It's as hard as it is to start flossing.

-Mitch Hedberg

Defining a Unit

class RandTests(unittest.TestCase):
    def test_evict(self):
        runner = vmsim.Runner()
        runner.run_single(Random,8,'12.trace')
        page_num_list = [1055,81758,24184,1141,201544,1187,28687,1182,1218,208986,24070]
        num_valid = 0
        for page_num in page_num_list:
            if runner.pTable.table[page_num].is_valid:
                num_valid += 1
        self.assertEqual(num_valid,8)

Dependency Injection

class ClockTests(unittest.TestCase):
    def test_evict_lru(self):
        runner = vmsim.Runner()
        runner.run_single(Clock,8,'clock.trace')
        rightful_victim = runner.pTable.table[81758]
        self.assertFalse(rightful_victim.is_valid)
        rightful_victim = runner.pTable.table[24184]

False Confidence

• A bad test is worse than no test
• False positive made me blind to a nasty bug

Yes

• Made the project more fun
• Exposed bad habits
• The cool kids do it

Learning

"The history of Western Civilization is a series of failures approximating success"

-Paolo Palmieri