How to Test an External API with 0 Mocks?

EZZINE Imen

PHP Symfony Backend Developer

 

@imenezzine

  @imenezzine1

 

I.Ezzine

Text

I.Ezzine

How to test an application consuming an external API?

I.Ezzine
  • Using the real API
  • Sandbox or dedicated environment
  • Containerizing the API locally
  • Using mocks


The possibilities

I.Ezzine

Using the real API

  • Billing per call

  • Is the external service unavailable?

  • Is the API subscription expired?

  • Limited to GET endpoints

 

I.Ezzine

Sandbox

  • Tests depending on external factors can fail if the sandbox stops responding.
  • Tests may run slower.
  • External calls.
I.Ezzine

Virtualization

 

  • Complex configuration
  • Potentially erroneous data
  • Limited case coverage
I.Ezzine

Using mocks

 

  • Decoupling
  • Risk of divergence between the Mock and the real API
  • Code cluttered with simulation details
  • Mock maintenance in case of API changes

   



I.Ezzine
use Symfony\Component\HttpClient\MockHttpClient;
use Symfony\Component\HttpClient\Response\MockResponse;

$responses = [
	new MockResponse ($body1, $info1),
    new MockResponse ($body2, $info2)
];

$client = new MockHttpClient($responses);
// responses in the same order as passed to MockHttpClient
$response1 = $client->request('...');
$response1 = $client->request('...');

How to define responses manually

use Symfony\Component\HttpClient\Response\JsonMockResponse;

$response = new JsonMockResponse([
    'foo' => 'bar',
]);
I.Ezzine
use PHPUnit\Framework\TestCase;
use Symfony\Component\HttpClient\MockHttpClient;
use Symfony\Component\HttpClient\Response\MockResponse;
use Psr\Log\NullLogger;

class AuthenticatedClientTest extends TestCase
{
    const BASE_AUTH_URI = 'https://example.com/auth';
    const BASE_REST_URI = 'https://example.com/rest';

    public function testRequestWithExpiredToken(): void
    {
        $authenticatedClient = new AuthenticatedClient(
            new MockHttpClient([
                function ($method, $url, $options): MockResponse {
                    $this->assertSame('POST', $method);
                    $this->assertSame(sprintf('%s/v2/token', self::BASE_AUTH_URI), $url);
                    $this->assertSame('{"grant_type":"client_credentials","client_id":"clientId","client_secret":"clientSecret","account_id":"accountId"}', $options['body']);

                    return new MockResponse('{"access_token":"expired_access_token"}', [
                        'http_code' => 200,
                    ]);
                },
                function ($method, $url, $options): MockResponse {
                    $this->assertSame('GET', $method);
                    $this->assertSame(sprintf('%s/whatever', self::BASE_REST_URI), $url);
                    $this->assertArrayHasKey('headers', $options);
                    $this->assertArrayHasKey('normalized_headers', $options);
                    $this->assertArrayHasKey('authorization', $options['normalized_headers']);
                    $this->assertArrayHasKey(0, $options['normalized_headers']['authorization']);
                    $this->assertSame('Authorization: Bearer expired_access_token', $options['normalized_headers']['authorization'][0]);

                    return new MockResponse('', [
                        'http_code' => 401,
                    ]);
                },
                function ($method, $url, $options): MockResponse {
                    $this->assertSame('POST', $method);
                    $this->assertSame(sprintf('%s/v2/token', self::BASE_AUTH_URI), $url);
                    $this->assertSame('{"grant_type":"client_credentials","client_id":"clientId","client_secret":"clientSecret","account_id":"accountId"}', $options['body']);
                    return new MockResponse('{"access_token":"new_access_token"}', [
                        'http_code' => 200,
                    ]);
                },
                function ($method, $url, $options): MockResponse {
                    $this->assertSame('GET', $method);
                    $this->assertSame(sprintf('%s/whatever', self::BASE_REST_URI), $url);
                    $this->assertArrayHasKey('headers', $options);
                    $this->assertArrayHasKey(0, $options['normalized_headers']['authorization']);
                    $this->assertSame('Authorization: Bearer new_access_token', $options['normalized_headers']['authorization'][0]);

                    return new MockResponse('', [
                        'http_code' => 201,
                    ]);
                },
            ]),
            'clientId',
            'clientSecret',
            self::BASE_AUTH_URI,
            'accountId',
            new NullLogger()
        );

        $authenticatedClient->request('GET', '/whatever');
    }
}
Testing Method Pros Cons
Real API - Realism of tests
- Result reliability		 - External dependency
- Cost
Sandbox - Test isolation
- Result reliability		 - Feature limitations
- External dependency
Mock -Total control
- Speed		 - Deviation from Reality
- Maintenability ²
Local API Containerization - Test isolation
- Realism of tests		 - Complex configuration
 - Local resource dependency		 
I.Ezzine

No tests?

I.Ezzine

What are we looking for?

  • Minimal code?
  • Less configuration or rather automation?
  • Reduced maintainability?
  • Complete decoupling?

 

                                                                                                            

I.Ezzine

"Don’t Mock What You Don’t Own"
                               the London School of TDD – the one that loooves mocks.

 

 

I.Ezzine
I.Ezzine

 Video Cassette Recorder

Existing solutions

 

I.Ezzine
I.Ezzine

How does it work?

I.Ezzine

Some Theory

I.Ezzine

How does it work?

Overwriting PHP functions (fopen(), file_get_contents(),
curl_exec()...)"

Retrieve all

details of the request

(metadata)

Execute the real request 

Retrieve all details of the response 

Storage

Request

Response

How do we proceed?

Basic single client

\VCR\VCR::configure()
->setCassettePath(base_path().'/tests/cassettes');

Storage path

\VCR\VCR::class;

Mode

//record
\VCR\VCR::configure()->setMode('new_episodes'); 
//record and replay
\VCR\VCR::configure()->setMode('once');

On

Eject

Off

\VCR\VCR::turnOn();

\VCR\VCR::turnOff();

\VCR\VCR::eject();

\VCR\VCR::configure()->setStorage('json');

Storage format

I.Ezzine

How do we proceed?

\VCR\VCR::configure()->enableRequestMatchers(['method', 'url', 'host']);
  • Request Matching
\VCR\VCR::configure()
    ->addRequestMatcher(
        'custom_matcher',
        function (\VCR\Request $first, \VCR\Request $second) {
            // custom request matching
            return true;
        }
    )
    ->enableRequestMatchers(['method', 'url', 'custom_matcher']);
  • Custom request matching
I.Ezzine
  • StreamWrapper
  • SoapClient
  • cUrl
\VCR\VCR::configure()->enableLibraryHooks(['curl'])

How do we proceed?

  • Library hooks
I.Ezzine

With PHPUnit?

class VCRTest extends TestCase
{
    /**
     * @vcr unittest_annotation_test
     */
    public function testInterceptsWithAnnotations()
    {
        // Requests are intercepted and stored into 
        // tests/fixtures/unittest_annotation_test
        $result = file_get_contents('http://google.com');

        $this->assertEquals(
        	'This is a annotation test dummy.',
             $result,
            'Call was not intercepted (using annotations).'
        );

        // VCR is automatically turned on and off.
    }
}

PHPUnit TestListener for PHP-VCR      phpunit-testlistener-vcr

Why should we use it?

  • Much faster tests
  • Independence from the API
  • HTTP request inspection
  • Deterministic tests
  • Monitoring changes in external APIs
  • Detection of unexpected HTTP requests
  • Reduced dependency on network calls
  • Simplified updating
  • No manual mocking
I.Ezzine

Concrete Example

I.Ezzine

How does it work?

  @vcr
  Scenario: retrieve a product
    Given 1 product with category "category_1" exist in the PIM
    When I retrieve the product from the pim with the root category "test"
    Then I should get the expected pim product
I.Ezzine
	
    Feature: creating an image via an API

	@vcr 
	Scenario: Using the recorded response from PHP-VCR
		Given I'm using PHP-VCR
		When I make a request to the create image API
		Then the response should contain the image URL
# config/tests/boostrap.php

use Symfony\Component\Dotenv\Dotenv;

require dirname(__DIR__).'/../vendor/autoload.php';
require __DIR__.'/VCRMatcher/CustomBodyMatcher.php';

\VCR\VCR::configure()
    ->addRequestMatcher(
        'custom_body_matcher',
        new CustomBodyMatcher()
    )
    ->enableRequestMatchers(['method', 'url', 'custom_body_matcher', 'post_fields'])
    ->enableLibraryHooks(['curl'])
    ->setCassettePath(dirname(__DIR__, 2).'/tests/Environment/IO/cassettes')
    ->setStorage('yaml')
    ->setMode('once');

// Turn on for Initializing VCR
\VCR\VCR::turnOn();

// Turn off for DI
\VCR\VCR::turnOff();

(new Dotenv())->bootEnv(dirname(__DIR__).'/../.env');

A bit of configuration!

I.Ezzine

The magic of PHP-VCR!

# tests/MockServerContexte.php

private bool $vcrOn = false;

/**
 * @BeforeScenario
 */
public function setupVCR(BeforeScenarioScope $scenarioEvent): void
{
    if ($scenarioEvent->getScenario()->hasTag('vcr')) {
        \VCR\VCR::turnOn();
        $this->vcrOn = true;
        $scenarioCassette = strtolower(
            str_replace(' ', '_', $scenarioEvent->getScenario()->getTitle())
        );
        $suiteName = $scenarioEvent->getSuite()->getName();
        \VCR\VCR::insertCassette($suiteName.'/'.$scenarioCassette.'.yaml');
    }
}
    
/**
 * @AfterScenario
 */
public function ejectCassette(): void
{
    if ($this->vcrOn) {
        \VCR\VCR::eject();
    }
    \VCR\VCR::turnOff();
}
I.Ezzine

How does it look?

-
    request:
        method: POST
        url: 'https://api.example.com/images'
        headers:
            Host: api.example.com
            Accept-Encoding: ''
            X-Auth-Client: your_auth_client
            X-Auth-Token: your_auth_token
            Accept: application/json
            User-Agent: GuzzleHttp/7
            Content-Type: 'multipart/form-data; boundary=your_boundary'
        body: !!binary |
            LS00NGIxNzhkZmRkZTFhNzQ1YWNjZjMwMTM4MmUzOWY4MTUxOTBjYzc1DQ
            pDb250ZW50LURpc3Bvc2l0aW9uOiBmb3JtLWRhdGE7IG5hbWU9ImltYWdlX
            2ZpbGUiOyBmaWxlbmFtZT0iaWNvbi5wbmciDQpDb250ZW50LUxlbmd0aDogMzEwDQp
            Db250ZW50LVR5cGU6IGltYWdlL3BuZw0KDQqJUE5HDQoaCgAAAA1JSERSAAAAEAAAABAIBgA
            AAB/z/2EAAAAJcEhZcwAACxIAAAsSAdLdfvwAAADoSURBVDiNpZPRDAAAElFTkSuQmCC
    response:
        status:
            http_version: '1.1'
            code: '200'
            message: OK
        headers:
            Content-Type: application/json
            Transfer-Encoding: chunked
            Connection: keep-alive
            Vary: Accept-Encoding
            X-Request-ID: 838d78668230cc576f6bd482d1229b45
            X-Rate-Limit-Requests-Left: '16101494'
            X-Rate-Limit-Time-Reset-Ms: '29999'
            X-Rate-Limit-Requests-Quota: '16101495'
            X-Rate-Limit-Time-Window-Ms: '30000'
            Strict-Transport-Security: 'max-age=31536000; includeSubDomains'
        body: '{"data":{"id":1,"product_id":2,"is_thumbnail":false,"sort_order":1,"description":"",
        "image_file":"y\/550\/icon__19355.png",
        "url_zoom":"https:\/\/cdn11.bigcommerce.com\/s-sxuyyapoo3\/products\/493\/images\
        /426\/icon__19355.1637747487.1280.1280.png?c=1","url_standard":"https:\/\/cdn11.bigcommerce.com\/s-

How to maintain this over time? 

  • Regular updates
  • Cassette versioning
  • Semantic naming of cassettes
I.Ezzine

PHP-VCR is a powerful tool for simplifying integration testing by recording and replaying HTTP requests.

Whether you opt to use its static methods or integrate it with PHPUnit, PHP-VCR can significantly enhance the stability and consistency of your tests by reducing reliance on external services.

By leveraging it wisely, you can make your tests more reliable and faster.

 

Conclusion

 

I.Ezzine

Thank you 

Questions?

I.Ezzine
Made with Slides.com