Symfony Full Stack Testing

Presented by

Symfony Experts

Welcome

Steven Rosato, B. Ing

President & Founder @ Solutions Majisti

Steven Rosato

Small company near Montreal (Boisbriand) focussed on development with Symfony and React/Redux.

/solutions.majisti

Today

Objectives

  • The Mentra of the Tester
  • TDD vs BDD
  • The Testing Pyramid (testing stack)
  • Symfony Testing
  • Frontend Testing (very brief)

The Mentra of the Tester

??

a Boob

a Guru

Don't be a boob

The Mentrality

  • Test first approach
  • Be prepared for serious conceptual design and discipline
  • Do not bite more than you can chew
  • Allow yourself some hack & slash or prototyping

Fixing a bug in production

You should test when...

You should not test when

  • You trash what you write in the short term
  • You design a prototype that does not become production
  • You are rich enough to clear your technical debt

Boob tip #1: You are poorer than you think

Your prototype will go in production

#no-bank

You should test when

Psst... Guru tip #1

  • Code maintainability is a concern
  • Client needs will evolve
  • Refactoring legacy to new structure (rewrite without testing is useless)

Your project will need maintenance

Guru tip #2

Your client's needs will evolve

Common Excuses For Not Testing

Psst... Guru tip #3

You do not have time because skipping automated testing is slower

I ain't got no time for that

We must deliver the project quickly. (RUSH, RUSH, RUSH)

This is only permanently temporary

It is "just" this. It will be simple. (they say)

We have a QA.

If you are the QA, just must leave NOW.

Think About It

  • How long do you spend testing manually?
  • Are those manual tests shared with your peers?
  • How long do you spend fixing bugs versus implementing features?
  • Are you afraid to refactor production code?
  • How long do you spend not understanding the intent of the code?
  • Do you have to test the same things over and over again?

Think About It

What if you actually took all this time to implement tests?

Are you saying tests do not take more time? Yes. It is actually faster.

You get code that is maintainable.

Maintainable code = more features = client happy

TDD vs BDD

TDD: Test Driven Development

BDD: Behaviour Driven Development

  • Write test first
  • Red/Green
  • Refactor
  • Emerged from TDD
  • Focusses on Object Behaviour rather than input/output only

DDD: Domain Driven Design approach

Why??

The Benefits

  • Enforces S.O.L.I.D and G.R.A.S.P principles
    • Single responsability: A Class should have only one reason to change
  • Forces you to go back to the drawing boards
  • Regression bugs covered
  • Refactor confidence
  • Lower long term costs

TDD Bad Example

<?php

class Counter
{
    public function testTick()
    {
        $counter = new Counter();
        $count = $counter->tick();
        $this->assertEquals($count, 1);
    }
}
  • Don't reverse implement

What's wrong?

BDD Good Example

<?php

class Counter
{
    public function shouldIncrementCounterByOneByDefault()
    {
        $counter = new Counter();
        
        $expectedCount = $counter->current() + 1;
        $this->checkThat($expectedCount, equalTo($counter->tick()));
    }
    
    public function shouldIncrementCounterByDefinedStep()
    {
        $step = 2;
        $counter = new Counter($step);
        
        $expectedCount = $counter->current() + $step;
        $this->checkThat($expectedCount, equalTo($counter->tick()));
    }
}

Object behaviour

No reverse implementation

Real World Example

TDD Bad Example

class GameTest extends UnitTest
{
    public function testGamePlayerWins()
    {
        $deck = m::mock(Deck::class);
        $deck->shouldReceive('draw')->once()->andReturn([new Card(10), new Card(7), 
            new Card(9), new Card(8), new Card(2)]);
        
        $dealerHand = m::mock(Hand::class);
        $playerHand = m::mock(Hand::class);
        
        $player = m::mock(Player::class);
        $dealer = m::mock(Dealer::class);
        
        $dealerHand->shouldReceive('addCards')->once()->with([new Card(10), new Card(7)]);
        $playerHand->shouldReceive('addCards')->once()->with([new Card(9), new Card(8)]);
        $playerHand->shouldReceive('addCards')->once()->with([new Card(2)]);
        
        $player->shouldReceive('setHand')->once()->with($playerHand);
        $dealer->shouldReceive('setHand')->once()->with($dealerHand);
        
        $game = new Game($dealerHand, $playerHand);
        $game->start();
        $game->hit($dealer, $deck, 2);
        $game->hit($player, $deck, 3);
        
        $this->assertSame($game->getWinner(), $player);
    }
}

What's wrong?

BDD Example continued...

Game has:

  • Too many responsibilities
    • It records information
    • It determines the winner
    • It coordinates the game
  • Reverse implementation
    • Refactoring without changing the object's behaviour will break tests, when it should not

True story bro. Real world case. Even though I wrote tests, I ended up with god objects because I did not think about the Object's Behaviour.

BDD Example continued...

class GameTest extends UnitTest
{
    public function shouldStartGame....????()
    {
        
    }

    public function shouldDetermineWinner...????()
    {

    }
}

BDD Example continued...

Thinking object's behaviour forces us to come back to the drawing boards and come up with a better solution for the current problem

BDD Example continued...

namespace Unit\Blackjack;

/**
 * @method Game uut()
 *
 * @property Dealer|m\MockInterface dealer
 * @property Player|m\MockInterface player
 */
class GameTest extends UnitTest
{
    protected $uut;

    protected function setUp()
    {
        $this->dealer = m::spy(Dealer::class);
        $this->player = m::spy(Player::class);

        parent::setUp();
    }

    protected function createUnitUnderTest()
    {
        $game = new Game();
        $game->setDealer($this->dealer);
        $game->setPlayer($this->player);

        return $game;
    }

BDD Example continued...

<?php

namespace Unit\Blackjack;

class GameTest extends UnitTest
{
    public function testTellsIfDealerWon()
    {
        $this->uut()->dealerWins();
        $this->verifyThat($this->uut()->hasDealerWon(), is(true));
        $this->verifyThat($this->uut()->hasPlayerWon(), is(false));
    }

    public function testTellsIfGameIsADraw()
    {
        $this->uut()->setIsDraw();
        $this->verifyThat($this->uut()->isDraw(), is(true));
    }

    public function testTellsIfPlayerWon()
    {
        $this->uut()->playerWins();
        $this->verifyThat($this->uut()->hasPlayerWon(), is(true));
        $this->verifyThat($this->uut()->hasDealerWon(), is(false));
    }
}

You understood BDD when

You didn't understand BDD when

  • You test input/output
  • You test the "How"
    • You reverse implement
  • You write code before tests
  • You test object behaviour
  • You test objects interactions with the UUT
  • Feature driven
  • You write easy to understand scenarios
  • Each object has its own responsibility

The Testing Pyramid

or Component

Available Tools in PHP

Mockery

Phake

AspectMock

Acceptance Testing

  • Selenium/PhantomJs Tests
  • Keep tests light
  • Can be used both on the Backend and Frontend (JavaScript)
  • Great for End-to-End testing
  • Great for legacy system testing (even Wordpress!)

Off load testing through the entire pyramid

Acceptance Testing

<?php
$I = new AcceptanceTester($scenario);
$I->wantTo('sign in');
$I->amOnPage('/login');
$I->fillField('username', 'Steven');
$I->fillField('password', 'qwerty');
$I->click('LOGIN');
$I->see('Welcome, Steven!');
?>

Exercise:

  • Page Object Pattern
  • Step Object Pattern

Acceptance Testing

@navigation
Feature:
  As a user
  I should be able to navigate the different pages of this site

  @homepage
  Scenario: Navigate the homepage
    Given I am on the homepage
    Then I take a screenshot or save last response named "homepage"

  @section
  Scenario: Navigate to a section page
    Given I visited the "Homepage"
    When I navigate to the "Section 1" section within the menu
    Then I should see a top stories widget
    And I should see a 2x2 layout widget
    And I should see a slideshow widget
    And I should see a multi layout widget
    And I should see a videos widget
    Then I take a screenshot or save last response named "section"

Functional Testing

  • Great for API testing
  • Command line testing
  • It's black-box testing, but at the technical level

Functional Testing

class ConfigDeploymentCommandCest extends DeploymentCommandCest
{
    const COMMAND_TO_RUN = 'ez:legacy:desktop-apps:config';

    /**
     * @param FunctionalTester $tester
     */
    public function seeTheModifiedConfigFileWithNewValues(FunctionalTester $tester)
    {
        $this->deleteTestFile($tester, DataConfiguration::scannerAppConfigOutput());

        $optionName = DesktopAppsDeploymentCommand::SCANNER_APP_OPTION_NAME;

        $tester->runCommand(static::COMMAND_TO_RUN, array('--' . $optionName => null));
        $tester->seeCommandOutput(sprintf(
            DesktopAppsDeploymentCommand::MESSAGE_DEPLOYMENT_SUCCESSFUL, 
            $optionName
        ));
        $tester->dontSeeAnException();
        $this->checkThatFileWasChanged(
            $tester, 
            DataConfiguration::scannerAppConfigInput(), 
            DataConfiguration::scannerAppConfigOutput()
        );
    }
}

Functional Testing

@functional @blackjack
Feature:
  As a player
  I should be able to play a game of Blackjack through the Symfony2 CLI

  Background:
    Given I trick the deck
    And I register the Blackjack command

  Scenario: Dealer wins by outscoring player
    Given the deck returns the following cards in a FILO order:
      #dealer cards
      |  6 |
      |  6 |

      #player cards
      |  5 |
      |  5 |

      |  5 |

    And I call "stand" when asked for my move
    When I run the blackjack game command
    Then the player score should be "10"
    And the dealer score should be "17"
    And the dealer should have won

Integration Testing

  • Great for component behaviour testing
  • Dependency Injection Container is your friend
  • You can test your edge cases here

Integration Testing

class BlackJackGameTest extends ComponentTest
{   
    public function testDealerLosesIfHeBusts()
    {
        $this->initialDealerCards(new Card(10), new Card(3));
        $this->initialPlayerCards(new Card(10), new Card(8));
        $this->nextPlayerCards([new Card(3)]);
        $this->nextDealerCards([new Card(9)]);

        $this->playerWillHitTimes(1);
        $game = $this->playEntireGame();

        $this->verifyThat($game->getDealer()->hasBusted(), is(true));
        $this->verifyThat($game->hasPlayerWon(), is(true));
    }

    public function testDealerShouldTryToStopAt17()
    {
        $this->initialDealerCards(new Card(10), new Card(3));
        $this->initialPlayerCards(new Card(10), new Card(8));
        $this->nextDealerCards([new Card(4), new Card(10)]);

        $game = $this->playEntireGame();

        $this->verifyThat($game->getDealer()->getBestScore(), is(equalTo(17)));
        $this->verifyThat($game->hasPlayerWon(), is(true));
    }

    //...
}

Integration Testing Continued...

class BlackJackGameTest extends ComponentTest
{   
    public function testGameDrawWhenBothPlayersHaveABlackjack()
    {
        $this->nextCardsFormABlackjack();
        $this->nextCardsFormABlackjack();

        $game = $this->coordinateGame();
        $this->verifyThat($game->isDraw(), is(true));
    }

    public function testPlayerWillUseBestScoreInOrderToWin()
    {
        $this->initialDealerCards(new Card(10), new Card(8));
        $this->initialDealerCards(new Card(5), new Card(Card::RANK_ACE));
        $this->nextDealerCards([new Card(3)]);

        $this->playerWillHitTimes(1);
        $game = $this->playEntireGame();

        $this->verifyThat($game->hasPlayerWon(), is(true));
    }
}

Unit Testing

  • Testing object interaction and behaviour
  • Dependency Injection is your friend
  • Mocking frameworks are your friends
  • Hamcrest

Unit Testing

<?php

namespace Unit\Blackjack;

/**
 * @method Dealer uut()
 *
 * @property Deck|m\MockInterface deck
 * @property Player|m\MockInterface player
 */
class DealerTest extends PlayerTest
{
    protected function setUp()
    {
        $this->player = m::mock(Player::class);

        $this->deck = m::mock(Deck::class);
        $this->deck->shouldReceive('draw')->andReturn(new Card())->byDefault();

        parent::setUp();
    }

    protected function createUnitUnderTest()
    {
        return new Dealer($this->deck);
    }  
}

Unit Testing

class DealerTest extends PlayerTest
{
    public function testCanGiveCardsToPlayer()
    {
        $player = m::mock(Player::class);
        $player->shouldReceive('receiveCard')
            ->times(2)
            ->with(anInstanceOf(Card::class));

        $this->uut()->hit($player, 2);
    }

    public function testDealerCannotDrawIfHeHasMoreThanSixteen()
    {
        $handCalculator = new HandCalculator();

        $this->deck->shouldReceive('draw')->andReturn(new Card(1));
        $this->uut()->receiveCards([
            new Card(10),
            new Card(6),
        ]);
        $this->uut()->calculateHand($handCalculator);

        $this->player->shouldReceive('getBestScore')->andReturn(18);

        $this->uut()->play($handCalculator);
        $this->verifyThat($this->uut()->getBestScore(), equalTo(17));
    }

    //todo: dealer can hit on a soft 17? Maybe add it as an optional rule (Rulebook object?)
}

How to choose which layer a test goes in?

  • Conceptualize top to bottom, but implement bottom to top
  • Behaviour Driven!
  • Domain Driven Design

Acceptance

Functional

Component

Unit

Virtual Browser

Blackbox

Technical
Command line
API

Set of classes

Repository

Database

Framework

Atomic
One per class

Objects Interactions

Real browser

UI

End-to-End

Third party libraries

The integration server counts in the entire stack

  • build process
  • deployment process
  • metrics thresholds
  • run the entire testing stack

Nightly builds

  • installation process
  • composer update
  • npm update

On commit build

phamtomjs + xfvb or SauceLabs ($$$) for Acceptance testing

We all wait to long to update our vendors

Symfony Testing

Integration Layer

  • The WebTestCase
  • Testing Controllers
  • Testing Forms
  • Testing Translations
  • Testing Configuration
  • Testing Mailing
  • Doctrine2/Database Testing

Functional Layer

  • Command line testing
  • API Testing

The WebTestCase from Symfony

Step in the right direction, but we are missing..

  • Command line testing
  • Database fixtures loading
  • Api testing

Command Line Testing

  • Do not create fat commands
  • Use a Command Helper
  • A command is essentially a controller!

Why?

  • A break in Symfony will break all commands
  • Can plug-in transactional database testing

What to Test?

  • Exceptions
  • The output
  • Interactions
  • The end result

API Testing

Stop manually testing using rest clients

Why?

  • Verify information you send to consumers
    • Response codes
  • Confident in refactoring (API versionning)
  • Easy to document breaking changes

What to Test?

  • A lot like a command line test
  • Response codes
  • Test different view formats (JSON, XML, HTML)
  • The end result
  • Idempotency

Use database caching

Testing Forms

Stop manually refreshing your browser

Why?

What to Test?

  • Form type
  • Added constraints
  • Translation keys
  • Do not need to manually test forms
    • validation
    • translation
    • data types

Use Symfony's TypeTestCase

Testing Controllers

  • Nothing to cover but the UI rendering
  • Other tests were made to ease out the testing of the controller
  • Covered by light acceptance testing or functional tests
  • You do not test controllers directly.

Testing Translations

  • JMSTranslationBundle
  • Command line test for extraction
    • ran only by the CI
  • Integration test for translated keys

Testing Configuration

When?

What to Test?

  • Custom constraints
  • Exception handling
  • Configuration choices
  • Business logic in your configuration

Testing Mailing

  • SwiftMailer Collector
/**
 * @author Steven Rosato <steven.rosato@majisti.com>
 * @group mail
 */
class MailSendingTest extends IntegrationTest
{
    /**
     * @test
     * @group mail.messageCount
     */
    public function itShouldSendAContactEmailToUserAndAdmin()
    {
        $this->fireMailer();
        $collector = $this->getCollector();

        $this->checkThat($collector, is(notNullValue()));
        $this->checkThat($collector->getMessageCount(), is(equalTo(2)));
    }

    /**
     * @return MessageDataCollector
     */
    private function getCollector()
    {
        return $this->getClient()->getProfile()->getCollector('swiftmailer');
    }
}

Testing Randomness

class Card
{
    public static function createRandom(): Card
    {
        $types = self::SUITS;

        $randomType = $types[rand(0, count($types) - 1)];
        $randomValue = rand(1, static::CARDS_PER_TYPE_COUNT);

        return new static($randomValue, $randomType);
    }
}

class CardTest extends UnitTest    
{    
    public function testCanCreateRandomCard()
    {
        $phpRandFunction = Test::func($this->getUutNamespace(), 
            'rand', Card::RANK_ACE);

        $card = $this->uut()->createRandom();
        $this->verifyThat($card, is(anInstanceOf(Card::class)));
        $this->verifyThat($card->getRank(), is(equalTo(Card::RANK_ACE)));

        $phpRandFunction->verifyInvoked([Card::RANK_ACE]);
    }
}

Database Testing

What to do? *

  • Use Doctrine2
  • Database caching with SQLite
    • Schema only loading
    • Full fixtures loading
  • Database transactions for every test case
  • Database snapshots

* Come talk to me for more info. This subject could cover an entire presentation by itself

Last but not least... Frontend Testing

  • Same approach (Acceptance, Functional, Integration, Unit)
    • Off load throughout the testing stack
    • Reactive pattern let's you test behavior of single components
    • Integration testing let's you test a full slider, for exemple
    • Protractor / chimp.js for acceptance. PhantomJs again!

Conclusion

  • Parallel testing
  • Automatic production deployment and rollback
  • Test performance with Vagrant and VMWare (VirtualBox is slow, even with NFS)

Stuff to still dig in...

Wow this is a lot!

  • Still fits for small and medium projects

Conclusion

Read us ranting

  • Symfony application development
  • Consultation with video recording
  • Friendly expert advice

Come blog with us

Symfony Experts

Quebecor Media. 20/09/2016. Full Stack Testing with Symfony.

By Solutions Majisti

Quebecor Media. 20/09/2016. Full Stack Testing with Symfony.

Full Stack Testing with Symfony

  • 1,021