Symfony Full Stack Testing
Presented by
Symfony Experts
Welcome
Steven Rosato, B. Ing
President & Founder @ Solutions Majisti
Steven Rosato
Small company near Montreal (Boisbriand) focussed on development with Symfony and React/Redux.
/solutions.majisti
Today
Objectives
- The Mentra of the Tester
- TDD vs BDD
- The Testing Pyramid (testing stack)
- Symfony Testing
- Frontend Testing (very brief)
The Mentra of the Tester
??
a Boob
a Guru
Don't be a boob
The Mentrality
- Test first approach
- Be prepared for serious conceptual design and discipline
- Do not bite more than you can chew
- Allow yourself some hack & slash or prototyping
Fixing a bug in production
You should test when...
You should not test when
- You trash what you write in the short term
- You design a prototype that does not become production
- You are rich enough to clear your technical debt
Boob tip #1: You are poorer than you think
Your prototype will go in production
#no-bank
You should test when
Psst... Guru tip #1
- Code maintainability is a concern
- Client needs will evolve
- Refactoring legacy to new structure (rewrite without testing is useless)
Your project will need maintenance
Guru tip #2
Your client's needs will evolve
Common Excuses For Not Testing
Psst... Guru tip #3
You do not have time because skipping automated testing is slower
I ain't got no time for that
We must deliver the project quickly. (RUSH, RUSH, RUSH)
This is only permanently temporary
It is "just" this. It will be simple. (they say)
We have a QA.
If you are the QA, just must leave NOW.
Think About It
- How long do you spend testing manually?
- Are those manual tests shared with your peers?
- How long do you spend fixing bugs versus implementing features?
- Are you afraid to refactor production code?
- How long do you spend not understanding the intent of the code?
- Do you have to test the same things over and over again?
Think About It
What if you actually took all this time to implement tests?
Are you saying tests do not take more time? Yes. It is actually faster.
You get code that is maintainable.
Maintainable code = more features = client happy
TDD vs BDD
TDD: Test Driven Development
BDD: Behaviour Driven Development
- Write test first
- Red/Green
- Refactor
- Emerged from TDD
- Focusses on Object Behaviour rather than input/output only
DDD: Domain Driven Design approach
Why??
The Benefits
- Enforces S.O.L.I.D and G.R.A.S.P principles
- Single responsability: A Class should have only one reason to change
- Forces you to go back to the drawing boards
- Regression bugs covered
- Refactor confidence
- Lower long term costs
TDD Bad Example
<?php
class Counter
{
public function testTick()
{
$counter = new Counter();
$count = $counter->tick();
$this->assertEquals($count, 1);
}
}
- Don't reverse implement
What's wrong?
BDD Good Example
<?php
class Counter
{
public function shouldIncrementCounterByOneByDefault()
{
$counter = new Counter();
$expectedCount = $counter->current() + 1;
$this->checkThat($expectedCount, equalTo($counter->tick()));
}
public function shouldIncrementCounterByDefinedStep()
{
$step = 2;
$counter = new Counter($step);
$expectedCount = $counter->current() + $step;
$this->checkThat($expectedCount, equalTo($counter->tick()));
}
}
Object behaviour
No reverse implementation
Real World Example
TDD Bad Example
class GameTest extends UnitTest
{
public function testGamePlayerWins()
{
$deck = m::mock(Deck::class);
$deck->shouldReceive('draw')->once()->andReturn([new Card(10), new Card(7),
new Card(9), new Card(8), new Card(2)]);
$dealerHand = m::mock(Hand::class);
$playerHand = m::mock(Hand::class);
$player = m::mock(Player::class);
$dealer = m::mock(Dealer::class);
$dealerHand->shouldReceive('addCards')->once()->with([new Card(10), new Card(7)]);
$playerHand->shouldReceive('addCards')->once()->with([new Card(9), new Card(8)]);
$playerHand->shouldReceive('addCards')->once()->with([new Card(2)]);
$player->shouldReceive('setHand')->once()->with($playerHand);
$dealer->shouldReceive('setHand')->once()->with($dealerHand);
$game = new Game($dealerHand, $playerHand);
$game->start();
$game->hit($dealer, $deck, 2);
$game->hit($player, $deck, 3);
$this->assertSame($game->getWinner(), $player);
}
}
What's wrong?
BDD Example continued...
Game has:
- Too many responsibilities
- It records information
- It determines the winner
- It coordinates the game
- Reverse implementation
- Refactoring without changing the object's behaviour will break tests, when it should not
True story bro. Real world case. Even though I wrote tests, I ended up with god objects because I did not think about the Object's Behaviour.
BDD Example continued...
class GameTest extends UnitTest
{
public function shouldStartGame....????()
{
}
public function shouldDetermineWinner...????()
{
}
}
BDD Example continued...
Thinking object's behaviour forces us to come back to the drawing boards and come up with a better solution for the current problem
BDD Example continued...
namespace Unit\Blackjack;
/**
* @method Game uut()
*
* @property Dealer|m\MockInterface dealer
* @property Player|m\MockInterface player
*/
class GameTest extends UnitTest
{
protected $uut;
protected function setUp()
{
$this->dealer = m::spy(Dealer::class);
$this->player = m::spy(Player::class);
parent::setUp();
}
protected function createUnitUnderTest()
{
$game = new Game();
$game->setDealer($this->dealer);
$game->setPlayer($this->player);
return $game;
}
BDD Example continued...
<?php
namespace Unit\Blackjack;
class GameTest extends UnitTest
{
public function testTellsIfDealerWon()
{
$this->uut()->dealerWins();
$this->verifyThat($this->uut()->hasDealerWon(), is(true));
$this->verifyThat($this->uut()->hasPlayerWon(), is(false));
}
public function testTellsIfGameIsADraw()
{
$this->uut()->setIsDraw();
$this->verifyThat($this->uut()->isDraw(), is(true));
}
public function testTellsIfPlayerWon()
{
$this->uut()->playerWins();
$this->verifyThat($this->uut()->hasPlayerWon(), is(true));
$this->verifyThat($this->uut()->hasDealerWon(), is(false));
}
}
You understood BDD when
You didn't understand BDD when
- You test input/output
- You test the "How"
- You reverse implement
- You write code before tests
- You test object behaviour
- You test objects interactions with the UUT
- Feature driven
- You write easy to understand scenarios
- Each object has its own responsibility
The Testing Pyramid
or Component
Available Tools in PHP
Mockery
Phake
AspectMock
Acceptance Testing
- Selenium/PhantomJs Tests
- Keep tests light
- Can be used both on the Backend and Frontend (JavaScript)
- Great for End-to-End testing
- Great for legacy system testing (even Wordpress!)
Off load testing through the entire pyramid
Acceptance Testing
<?php
$I = new AcceptanceTester($scenario);
$I->wantTo('sign in');
$I->amOnPage('/login');
$I->fillField('username', 'Steven');
$I->fillField('password', 'qwerty');
$I->click('LOGIN');
$I->see('Welcome, Steven!');
?>
Exercise:
- Page Object Pattern
- Step Object Pattern
Acceptance Testing
@navigation
Feature:
As a user
I should be able to navigate the different pages of this site
@homepage
Scenario: Navigate the homepage
Given I am on the homepage
Then I take a screenshot or save last response named "homepage"
@section
Scenario: Navigate to a section page
Given I visited the "Homepage"
When I navigate to the "Section 1" section within the menu
Then I should see a top stories widget
And I should see a 2x2 layout widget
And I should see a slideshow widget
And I should see a multi layout widget
And I should see a videos widget
Then I take a screenshot or save last response named "section"
Functional Testing
- Great for API testing
- Command line testing
- It's black-box testing, but at the technical level
Functional Testing
class ConfigDeploymentCommandCest extends DeploymentCommandCest
{
const COMMAND_TO_RUN = 'ez:legacy:desktop-apps:config';
/**
* @param FunctionalTester $tester
*/
public function seeTheModifiedConfigFileWithNewValues(FunctionalTester $tester)
{
$this->deleteTestFile($tester, DataConfiguration::scannerAppConfigOutput());
$optionName = DesktopAppsDeploymentCommand::SCANNER_APP_OPTION_NAME;
$tester->runCommand(static::COMMAND_TO_RUN, array('--' . $optionName => null));
$tester->seeCommandOutput(sprintf(
DesktopAppsDeploymentCommand::MESSAGE_DEPLOYMENT_SUCCESSFUL,
$optionName
));
$tester->dontSeeAnException();
$this->checkThatFileWasChanged(
$tester,
DataConfiguration::scannerAppConfigInput(),
DataConfiguration::scannerAppConfigOutput()
);
}
}
Functional Testing
@functional @blackjack
Feature:
As a player
I should be able to play a game of Blackjack through the Symfony2 CLI
Background:
Given I trick the deck
And I register the Blackjack command
Scenario: Dealer wins by outscoring player
Given the deck returns the following cards in a FILO order:
#dealer cards
| 6 |
| 6 |
#player cards
| 5 |
| 5 |
| 5 |
And I call "stand" when asked for my move
When I run the blackjack game command
Then the player score should be "10"
And the dealer score should be "17"
And the dealer should have won
Integration Testing
- Great for component behaviour testing
- Dependency Injection Container is your friend
- You can test your edge cases here
Integration Testing
class BlackJackGameTest extends ComponentTest
{
public function testDealerLosesIfHeBusts()
{
$this->initialDealerCards(new Card(10), new Card(3));
$this->initialPlayerCards(new Card(10), new Card(8));
$this->nextPlayerCards([new Card(3)]);
$this->nextDealerCards([new Card(9)]);
$this->playerWillHitTimes(1);
$game = $this->playEntireGame();
$this->verifyThat($game->getDealer()->hasBusted(), is(true));
$this->verifyThat($game->hasPlayerWon(), is(true));
}
public function testDealerShouldTryToStopAt17()
{
$this->initialDealerCards(new Card(10), new Card(3));
$this->initialPlayerCards(new Card(10), new Card(8));
$this->nextDealerCards([new Card(4), new Card(10)]);
$game = $this->playEntireGame();
$this->verifyThat($game->getDealer()->getBestScore(), is(equalTo(17)));
$this->verifyThat($game->hasPlayerWon(), is(true));
}
//...
}
Integration Testing Continued...
class BlackJackGameTest extends ComponentTest
{
public function testGameDrawWhenBothPlayersHaveABlackjack()
{
$this->nextCardsFormABlackjack();
$this->nextCardsFormABlackjack();
$game = $this->coordinateGame();
$this->verifyThat($game->isDraw(), is(true));
}
public function testPlayerWillUseBestScoreInOrderToWin()
{
$this->initialDealerCards(new Card(10), new Card(8));
$this->initialDealerCards(new Card(5), new Card(Card::RANK_ACE));
$this->nextDealerCards([new Card(3)]);
$this->playerWillHitTimes(1);
$game = $this->playEntireGame();
$this->verifyThat($game->hasPlayerWon(), is(true));
}
}
Unit Testing
- Testing object interaction and behaviour
- Dependency Injection is your friend
- Mocking frameworks are your friends
- Hamcrest
Unit Testing
<?php
namespace Unit\Blackjack;
/**
* @method Dealer uut()
*
* @property Deck|m\MockInterface deck
* @property Player|m\MockInterface player
*/
class DealerTest extends PlayerTest
{
protected function setUp()
{
$this->player = m::mock(Player::class);
$this->deck = m::mock(Deck::class);
$this->deck->shouldReceive('draw')->andReturn(new Card())->byDefault();
parent::setUp();
}
protected function createUnitUnderTest()
{
return new Dealer($this->deck);
}
}
Unit Testing
class DealerTest extends PlayerTest
{
public function testCanGiveCardsToPlayer()
{
$player = m::mock(Player::class);
$player->shouldReceive('receiveCard')
->times(2)
->with(anInstanceOf(Card::class));
$this->uut()->hit($player, 2);
}
public function testDealerCannotDrawIfHeHasMoreThanSixteen()
{
$handCalculator = new HandCalculator();
$this->deck->shouldReceive('draw')->andReturn(new Card(1));
$this->uut()->receiveCards([
new Card(10),
new Card(6),
]);
$this->uut()->calculateHand($handCalculator);
$this->player->shouldReceive('getBestScore')->andReturn(18);
$this->uut()->play($handCalculator);
$this->verifyThat($this->uut()->getBestScore(), equalTo(17));
}
//todo: dealer can hit on a soft 17? Maybe add it as an optional rule (Rulebook object?)
}
How to choose which layer a test goes in?
- Conceptualize top to bottom, but implement bottom to top
- Behaviour Driven!
- Domain Driven Design
Acceptance
Functional
Component
Unit
Virtual Browser
Blackbox
Technical
Command line
API
Set of classes
Repository
Database
Framework
Atomic
One per class
Objects Interactions
Real browser
UI
End-to-End
Third party libraries
The integration server counts in the entire stack
- build process
- deployment process
- metrics thresholds
- run the entire testing stack
Nightly builds
- installation process
- composer update
- npm update
On commit build
phamtomjs + xfvb or SauceLabs ($$$) for Acceptance testing
We all wait to long to update our vendors
Symfony Testing
Integration Layer
- The WebTestCase
- Testing Controllers
- Testing Forms
- Testing Translations
- Testing Configuration
- Testing Mailing
- Doctrine2/Database Testing
Functional Layer
- Command line testing
- API Testing
The WebTestCase from Symfony
Step in the right direction, but we are missing..
- Command line testing
- Database fixtures loading
- Api testing
Command Line Testing
- Do not create fat commands
- Use a Command Helper
- A command is essentially a controller!
Why?
- A break in Symfony will break all commands
- Can plug-in transactional database testing
What to Test?
- Exceptions
- The output
- Interactions
- The end result
API Testing
Stop manually testing using rest clients
Why?
- Verify information you send to consumers
- Response codes
- Confident in refactoring (API versionning)
- Easy to document breaking changes
What to Test?
- A lot like a command line test
- Response codes
- Test different view formats (JSON, XML, HTML)
- The end result
- Idempotency
Use database caching
Testing Forms
Stop manually refreshing your browser
Why?
What to Test?
- Form type
- Added constraints
- Translation keys
- Do not need to manually test forms
- validation
- translation
- data types
Use Symfony's TypeTestCase
Testing Controllers
- Nothing to cover but the UI rendering
- Other tests were made to ease out the testing of the controller
- Covered by light acceptance testing or functional tests
- You do not test controllers directly.
Testing Translations
- JMSTranslationBundle
- Command line test for extraction
- ran only by the CI
- Integration test for translated keys
Testing Configuration
When?
What to Test?
- Custom constraints
- Exception handling
- Configuration choices
- Business logic in your configuration
Testing Mailing
- SwiftMailer Collector
/**
* @author Steven Rosato <steven.rosato@majisti.com>
* @group mail
*/
class MailSendingTest extends IntegrationTest
{
/**
* @test
* @group mail.messageCount
*/
public function itShouldSendAContactEmailToUserAndAdmin()
{
$this->fireMailer();
$collector = $this->getCollector();
$this->checkThat($collector, is(notNullValue()));
$this->checkThat($collector->getMessageCount(), is(equalTo(2)));
}
/**
* @return MessageDataCollector
*/
private function getCollector()
{
return $this->getClient()->getProfile()->getCollector('swiftmailer');
}
}
Testing Randomness
- Using AspectMock from Codeception
class Card
{
public static function createRandom(): Card
{
$types = self::SUITS;
$randomType = $types[rand(0, count($types) - 1)];
$randomValue = rand(1, static::CARDS_PER_TYPE_COUNT);
return new static($randomValue, $randomType);
}
}
class CardTest extends UnitTest
{
public function testCanCreateRandomCard()
{
$phpRandFunction = Test::func($this->getUutNamespace(),
'rand', Card::RANK_ACE);
$card = $this->uut()->createRandom();
$this->verifyThat($card, is(anInstanceOf(Card::class)));
$this->verifyThat($card->getRank(), is(equalTo(Card::RANK_ACE)));
$phpRandFunction->verifyInvoked([Card::RANK_ACE]);
}
}
Database Testing
What to do? *
- Use Doctrine2
- Database caching with SQLite
- Schema only loading
- Full fixtures loading
- Database transactions for every test case
- Database snapshots
* Come talk to me for more info. This subject could cover an entire presentation by itself
Last but not least... Frontend Testing
- Same approach (Acceptance, Functional, Integration, Unit)
- Off load throughout the testing stack
- Reactive pattern let's you test behavior of single components
- Integration testing let's you test a full slider, for exemple
- Protractor / chimp.js for acceptance. PhantomJs again!
Conclusion
- Parallel testing
- Automatic production deployment and rollback
- Test performance with Vagrant and VMWare (VirtualBox is slow, even with NFS)
Stuff to still dig in...
Wow this is a lot!
- Still fits for small and medium projects
Conclusion
Read us ranting
- Symfony application development
- Consultation with video recording
- Friendly expert advice
Come blog with us
Symfony Experts
Quebecor Media. 20/09/2016. Full Stack Testing with Symfony.
By Solutions Majisti
Quebecor Media. 20/09/2016. Full Stack Testing with Symfony.
Full Stack Testing with Symfony
- 1,021