Your code is probably wrong

The Market

demands, and rewards functionality, and reliability 

A primary challenge in producing a functioning,

reliable application is battling incorrect code

Strategies 

• Functionality

  • Type Systems

  • Automated Testing

  • Static analysis

  • Human review

• Reliability

  • Build error insensitive run-times
  • Force developers to provide error handlers
  • More correct code
  • Automated Testing
  • Human review

JavaScript's Answer

• Functionality

  • Human review

• Reliability

  • Build error insensitive run-times
  • Human review

JavaScript is a Train-wreck for producing reliable functionality

From one extreme to the other...

describe("email validator", function(){

  it("should bail if the first argument is not a sting", function(){
    expect(isValidEmail(3)).to.be.false;
  });

  it("should validate", function(){
    expect(isValidEmail("foo@bar.com")).to.be.true;
  });

  it("should validate an array of strings if provided", function(){
    expect(isValidEmail(["foo@bar.com", "baz@foo.net"])).to.be.true;
  });  

});

JavaScript Unit Tests

a manual type system

expect(isValidEmail(3)).to.be.false;

swallowing bugs?

Why would the application
EVER pass a Number to this function?

expect(isValidEmail(3)).to.throw.an.error
  ("isValidEmail expected a string, got a Number");

or risk runtime crashes?

expect(isValidEmail(["foo@bar.com", "baz@foo.net"]))
  .to.be.true;

what did we learn here?

A subset of the problem space

expect(isValidEmail(
   ["foo@bar.com", "baz@foo.net", "jim.jeff@jefferson.net", "cindy@cindyweb.it", ...]))

how many cases are 'enough'?

JavaScript Unit Tests

JavaScript Unit Tests

The imagination of the developer is limited. Manually providing coverage of the problem space is

not feasible.

A subset of the problem space

[ "foo@bar.com"
, "baz@foo.net"
, "jim.jeff@jefferson.net"
, "cindy@cindyweb.it"
, ...]))

//missing
"erica@统统.c"

Property Testing

More coverage for less

Generate perturbing cases

instead of specifying them manually.

Doing so requires that the developer reason about the code
in terms of generalized properties instead of specifics.

it("should place the third argument as the age of the User object", function(){

  var age = 5;
  var user = new User("Jennifer", "female", age);

  expect(user.age).to.equal(age);

});

Converting a Unit Test to

a Property Test

Why 5? Its a magic number, which is common in unit testing.

it("should place the third argument as the age of the User object", function(){

  var age = _.random(0, 10000);
  var user = new User("Jennifer", "female", age);

  expect(user.age).to.equal(age);

});

Using a random number is just as valid
as the previous unit test. But the property is now clear.

it("should place the third argument as the age of the User object", function(){
  var age  = null;
  var user = null;

  for(var i = 0; i < 100; i++){  
    age  = _.random(0, 10000);
    user = new User("Jennifer", "female", age);
    expect(user.age).to.equal(age);
  }

});

Now we have a property test, any Number passed for age should work, and we are covering more of the problem space.

is `random` really the best we can do here?

Property Testing 

In JavaScript

• JSCheck

• test-check

Looking at 2 JavaScript Property Testing Libraries 

JSC.test(
    "Third argument is age",

    function (verdict, age) {
        return verdict(new User("jennifer", "female", age));
    },

    [ JSC.integer(10) ],

    function (age, user) {
       return user.age == age;
    }

);

JSCheck

testcheck.check(testcheck.property(

  [gen.int],

  function (age) {
    var user = new User("jennifer", "female", age);
    return user.age == age;
  }

));

test-check

JavaScript is Dynamically Typed,

so the problem space is

the entire type universe.

Property tests do nothing to ensure that developers gracefully handle
type mis-matches.

expect(isValidEmail(3)).to.be.false;

swallowing bugs?

expect(isValidEmail(3)).to.throw.an.error
  ("isValidEmail expected a string, got a Number");

or risk runtime crashes?

The best option may be to conditionally error in development environments and write comprehensive end to end tests.

PureScript

PureScript is a small strongly typed programming language

that compiles to JavaScript.

PureScript

PureScript's Hindley–Milner style type system is by far the best type system available for JavaScript run-times.

(no I am not prepared to substantiate my opinion here)

QuickCheck

The canonical example of a property testing library is Haskell's QuickCheck. QuickCheck is available
in PureScript as well.

quickCheck will use Math.random to generate 100 perturbing numbers to verify this property holds.

And since PureScript is typed, the problem space is cleanly limited. Developers do not need to

handle mis-matched types.

it "the third argument should be age on the user object" $

  let 
    checkAge :: Number -> Boolean
    checkAge n = (newUser "jennifer" "female" n).age == n

  in quickCheck checkAge

So we just learned our code is wrong.

So we use <?> to convert our Boolean into a Result containing

a helpful error message. 

it "the third argument should be age on the user object" $

  let 
    checkAge :: Number -> Result
    checkAge n = let user = newUser "jennifer" "female" n 
    in user.age == n
    <?> "Age test, it failed, FAILED!"
    <> "\n when n = " <> show n 
    <> "\n newUser produced : "
    <> "\n " <> show user

  in quickCheck checkAge

(<?>) :: Boolean -> String -> Result

What about Custom Types?

instance userArb :: Arbitrary User where 
  arbitrary = newUser <$> arbitrary 
                      <*> arbitrary 
                      <*> arbitrary

checkYounger :: User -> User -> Boolean
checkYounger u@(User {age = a}) u'@(User {age = a'}) = 

    let res = younger u u'
    in if a < a' then res == true else res == false

quickCheck checkYounger 

Now QuickCheck knows how to make Perturbing Users

How about Arrows?

instance userCoarb :: CoArbitrary User where
  coarbitrary (User {age = a}) = coarbitrary a 

checkReadWriteFancyRef :: (User -> User) -> User -> Boolean
checkReadWriteFancyRef arrow user = extract $ do 
    userRef <- newFancyRef user
    writeFancyRef userRef arrow
    user' <- readFancyRef userRef
    return $ user == user'

quickCheck checkReadWriteFancyRef 

Now QuickCheck can make Arbitrary Arrows with User as the Domain or CoDomain!