How to write your own (fake) Vulkan Driver

Taking Mocking to the Moon: A testing story

By Charles Giessen

Who am I?

Charles Giessen

 

I work at LunarG, working on tooling and ecosystem for the Vulkan API

 

I primarily focus on the Vulkan-Loader but also work on Vulkaninfo and vk-bootstrap

 

You might remember me from a 2019 lightning talk about Vulkaninfo...

What is Vulkan?

Its a low level, explicit graphics API

 

Think of OpenGL or DirectX. Same thing, different API

 

Made by the Khronos Group, who also make OpenGL

What is the Vulkan-Loader?

  • Library which is the heart of the Vulkan Ecosystem
  • every (desktop) Vulkan application goes through the loader

Components of Vulkan

Applications - Makes the Vulkan API calls

 

Layers - optional plugins to the API, intercept function calls to do stuff like validation, recording, debugging, etc

 

Drivers - The actual graphics driver that executes all the API calls on a real Vulkan "implementation".

--Problem--

Vulkan-Loader has minimal test coverage

Specifically:

  • Tests are flaky - depend on host system state
    • Use drivers on the system, and work around them
  • They aren't authoritative
    • E.g. call function to get a value, call the same function again and check that the same value is returned

Difficulties

  • Vulkan-Loader codebase is C99, not C++ sigh
    • Test code is C++ fortunately
  • Fragile - little structure in the code
    • Can't add unit tests without possibly introducing bugs
  • Extremely integrated into the host system
    • Filesystem, registry, env-vars, OS calls left and right
    • Current tests modify host system in non-destructive ways, very limiting
    • But destructive host modification is a no-go, would interfere with other running Vulkan apps

Solution!

Mock the world!

Wait, wrong mocking

"mocking" the world

  1. Find all possible inputs to the Vulkan-Loader
    • Filesystem
    • Registry - Windows only
    • Environment Variables
    • OS specific calls
    • Drivers
    • Layers
  2. Intercept and/or fake them
  3. ...
  4. Profit!
  • Black box is the 'FrameworkEnvironment'.  It coordinates the whole architecture.
  • Green is the 'Test Cases'. These setup system state then run tests against them
  • Brown are Mock Layers
  • Blue are Mock Drivers
    • Which have Mock Physical Devices

 

 

FrameworkEnvironment

  • Intercepts & modifies all the system state, including
    • Filesystem & access to it
    • Windows Registry
    • Windows Driver API's (DXGI, D3DKMT)
    • Environment Variables
    • Elevated process state (is it running as sudo)

Windows Specific stuff

  • Registry interception
    • `RegOverridePredefKey` is like chroot but for the registry
    • Lots of manual work to recreate the registry that the loader expects
  • Windows Driver API's
    • Loader calls into Windows API's for information about drivers
    • Must recreate that by intercepting the functions & providing the desired information
    • Uses the Detours library to do the DLL injection
      • Detours uses makefiles for some odd reason...

Linux/MacOS specific stuff

  • Only need to redirect filesystem calls
    • opendir, access, and fopen are the only calls used
    • Tests still use the filesystem, the calls just return the data we want them to.
  • File search logic is complicated, using XDG_*** env-vars as a source
    • Must clear those out and set them to known values
  • Function interception works with the dynamic linker out of the box
    • MacOS needs magic incantation to work
struct Interposer {
    const void* shim_function;
    const void* underlying_function;
};
__attribute__((used)) static Interposer _interpose_opendir 
	__attribute__((section("__DATA,__interpose"))) = 
    	{VOIDP_CAST(my_opendir), VOIDP_CAST(opendir)};

Mock Driver aka TestDriver

  • Shared Library which implements the 'Driver Interface' the loader expects
    • test_driver.h
    • test_driver.cpp
  • Need a way to talk to the TestDriver from the test
    • Load a function pointer from TestDriver which grants direct access to the data structures used
  • Highly configurable
    • Tests can set modify the TestDriver in just about every way, except....

Binary Exports

  • Loader expects a Driver to export certain functions
    • Which are exported changes per interface version
  • Thus, multiple driver binaries need to exist for complete testing
  • CMake is used to stamp out the variations
  • What is exported is controlled by compiler options

Mock Layer aka TestLayer

  • Almost identical to TestDriver
  • Also has function to get & set state
  • Also has multiple binary exports

Test Code

  • Basic outline
    • Create a framework
    • Set the state
    • Call Vulkan API functions
    • Check results
  • Setup a 'Physical Device' in the mock
  • Add a layer
  • Check if that layer can be found
TEST(CreateInstance, LayerPresent) {
    FrameworkEnvironment env{};
    const char* layer_name = "TestLayer";
    env.add_explicit_layer(
        ManifestLayer{}.add_layer(
            ManifestLayer::LayerDescription{}
            .set_name(layer_name)
            .set_lib_path(TEST_LAYER_PATH_EXPORT_VERSION_2)),
        "test_layer.json");

    InstWrapper inst{env.vulkan_functions};
    inst.create_info.add_layer(layer_name);
    inst.CheckCreate();
    
    uint32_t layer_count = 0;
    ASSERT_EQ(VK_SUCCESS, env.vulkan_functions.
    	vkEnumerateInstanceLayerProperties(&layer_count, nullptr));
    ASSERT_EQ(layer_count, 1);
}

Example Test

Thanks!

Questions?

Made with Slides.com