Debug your DirectX application using Visual Studio

As every programmer does, I found myself in a situation where the dumb computer doesn’t do what I want it to do. During the early days of debugging my graphics applications, I was coloring things in particular colors to make sense of the debugging situation I was in. For example for a simple case of displaying textures on objects, to understand why a texture wouldn’t render on a particular object, I’d first modify the pixel shader to render UV coordinates to see if I was getting the data from vertex buffer correctly.

Scene Screenshot

Why my cubes aren't lit?

A lot of the times I wondered ‘wouldn’t it be nice if I would debug this code just like I step through in Visual Studio?’, until I have used a proper GPU debugger for console development during my internship. Seeing the contents of every buffer, every uniform variable (or shader constants if you come from DirectX background) profiling, stepping through code for a specific frame; all of these were possible with the debugger. After ~2 months of console development and the acquired experience, I decided to write a DirectX11 Renderer. Since I don’t have access to the capabilities of NVIDIA Nsight as I am on the Team Red and currently use an R9 380 and I am not really happy with what AMD has to offer for GPU debugging, I wondered what Microsoft has to offer for my situation. After all I was developing on an operating system, a development environment and a graphics API all made by the same company.

AMD Debugger

GPU Perf unfortunately crashes if the smallest thing goes wrong. Not really helpful :(

Pre-requisites

To be able to debug your graphics application through Visual Studio 2015 you’ll need a few things.

  • admin access to run the debugger on your application
  • enable debugging on your graphics device when creating it

The latter of the requirements can be done when initializing the Direct3D device by setting the D3D11_CREATE_DEVICE_DEBUG flag when calling D3D11CreateDeviceAndSwapChain() or D3D11CreateDevice().

// Create the swap chain, Direct3D device, and Direct3D device context.
result = D3D11CreateDeviceAndSwapChain( NULL, 
                    D3D_DRIVER_TYPE_HARDWARE, 
                    NULL, 
                    D3D11_CREATE_DEVICE_DEBUG,  // <-- enable debug information
                    &featureLevel, 
                    1,
                    D3D11_SDK_VERSION, 
                    &swapChainDesc, 
                    &m_swapChain, 
                    &m_device, NULL, &m_deviceContext );

Debug Scenario

As seen in the screenshot above, the scene consists of a room, in the middle of which there are an array of cubes that have a rendering issue. Here’s the code responsible for this:

  • Material represents the data that defines the material properties
  • Model represents a mesh and a material - the model to render
  • GameObject encapsulates a game object data, holding together its transform and model data
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
class Material
{
    Color       color;
    float       alpha;  
    XMFLOAT3    specular;
    float       shininess;

    Texture     diffuseMap;
    Texture     normalMap;

    static const Material jade, ruby, bronze, gold;
    Material();	// sets default material values (white diffuse, no textures)
    void SetMaterialConstants(Renderer* renderer) const    // sends the material data to GPU
    {			    // todo: use smart pointers
        renderer->SetConstant3f("diffuse", color.Value());
        renderer->SetConstant1f("shininess", shininess);
        renderer->SetConstant1f("alpha", alpha);
        renderer->SetConstant3f("specular", specular);
        renderer->SetConstant1f("isDiffuseMap", diffuseMap.id == -1 ? 0.0f : 1.0f);
        if (diffuseMap.id>=0) renderer->SetTexture("gDiffuseMap", diffuseMap.id);
        renderer->SetConstant1f("isNormalMap", normalMap.id == -1 ? 0.0f : 1.0f);
        if (normalMap.id>=0) renderer->SetTexture("gNormalMap", normalMap.id);
    }
};

class Model
{
    int         m_mesh; 
    Material    m_material;
};

class GameObject
{
    Transform   m_transform;
    Model       m_model;
};


  • A SceneManager holds the scene data and manages the initialization, updating and rendering of the scene objects.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
void SceneManager::InitializeObjectArrays()
{
    const int   row = 5,
                col = 5;
    const float r = 3.5f;

    // grid arrangement ( (row * col) cubes that are 'r' apart from each other )
    for (int i = 0; i < row; ++i)
    {
        for (int j = 0; j < col; ++j)
        {
            GameObject cube;

            // set transform
            float x, y, z;  // position
            x = i * r;      y = 5.0f;   z = j * r;
            cube.m_transform.SetPosition(x, y, z);

            // set material
            cube.m_model.m_mesh = MESH_TYPE::CUBE;
            cube.m_model.m_material = Material();   // default

            cubes.push_back(cube);
        }
    }
}

void SceneManager::Render(const XMMATRIX& view, const XMMATRIX& proj) 
{
    m_skydome.Render(m_renderer, view, proj);
    RenderLights(view, proj);
    RenderAnimated(view, proj);
    RenderBuilding(view, proj);
    RenderCentralObjects(view, proj);   // <-- renders the cubes
}

void SceneManager::RenderCentralObjects(const XMMATRIX& view, const XMMATRIX& proj) 
{
    // set shader and send constants
    m_renderer->SetShader(m_selectedShader);    //<-- clears all the buffers on GPU when a new shader is set
    if(m_selectedShader == m_renderData->phongShader) SendLightData();
    m_renderer->SetConstant4x4f("view", view);
    m_renderer->SetConstant4x4f("proj", proj);
    m_renderer->SetConstant3f("cameraPos", m_camera->GetPositionF());

    for (const auto& cube : cubes)
    {
        m_renderer->SetBufferObj(cube.m_model.m_mesh);
        XMMATRIX world = cube.m_transform.WorldTransformationMatrix();
        XMMATRIX nrm   = cube.m_transform.NormalMatrix(world);
        m_renderer->SetConstant4x4f("world", world);
        m_renderer->SetConstant4x4f("nrmMatrix", nrm);
        m_renderer->Apply();
        m_renderer->DrawIndexed();
    }
}


  • And a forward rendering phong lighting shader forward_phong.hlsl
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
struct PSIn
{
    float4 position : SV_POSITION;
    float3 worldPos : POSITION;
    float3 normal   : NORMAL;
    float3 tangent  : TANGENT;
    float2 texCoord : TEXCOORD4;
};

struct Surface
{
    float3 N;
    float3 diffuseColor;
    float3 specularColor;
    float shininess;
};

// render constants such as cameraPos, view, proj, etc...

float4 PSMain(PSIn In) : SV_TARGET
{
    // lighting & surface parameters
    float3 N = normalize(In.normal);
    float3 T = normalize(In.tangent);
    float3 V = normalize(cameraPos - In.worldPos);
    const float ambient = 0.005f;
    Surface s;
    s.N             =   (isNormalMap       ) * UnpackNormals(In.texCoord, N, T) + 
                        (1.0f - isNormalMap) * N;
    // -------------- objects color is read from texture if `isDiffuseMap == 1.0f` or set to white -----------
    s.diffuseColor  = diffuse *  ( isDiffuseMap          * gDiffuseMap.Sample(samAnisotropic, In.texCoord).xyz
                                 + (1.0f - isDiffuseMap) * float3(1,1,1));
    // -------------------------------------------------------------------------------------------------------
    s.specularColor = specular;
    s.shininess = shininess * SHININESS_ADJUSTER;

    // illumination
    float3 Ia = s.diffuseColor * ambient;   // ambient
    float3 IdIs = float3(0.0f, 0.0f, 0.0f); // diffuse & specular
    for (int i = 0; i < lightCount; ++i)    // POINT Lights
        IdIs += Phong(lights[i], s, V, In.worldPos) * Attenuation(lights[i].attenuation, 
                                                                  length(lights[i].position - In.worldPos));
    for (int j = 0; j < spotCount; ++j)     // SPOT Lights
        IdIs += Phong(spots[j], s, V, In.worldPos) * Intensity(spots[j], In.worldPos);

    float4 outColor = float4(Ia + IdIs, 1);
    return outColor;
}

Did you already catch the mistake? No? Allright, let’s fire the shiny debugger up and see what’s going on.



Graphics Diagnostics

Go to Debug->Graphics->Start Graphics Debugging or press Alt+F5, and confirm admin rights when prompted.

Run Debugger

You’ll see a diagnostics session running behind your application. After getting a good angle that captures the problem,

  1. press the Capture Frame button
  2. then stop the simulation

CaptureFrameNStop

Visual Studio will capture pretty much every event during one frame (or however many you set it), and will prepare a nice report for you do start a comfortable debugging session.

Debug Report

Frame Capture Report

Before diving into figuring out what’s wrong directly, let’s have a look around.


Event List

Since the View is set to GPU Work, the event list will list all the draw calls, clears and similar tasks.

Event List & Call Stack

Event list shows all the resources bound. Click on `obj:#` to view one

Additionally, if you change the View setting to Timeline, you will see all the Direct3D calls and will be able to look into each and every one of data structure used by the Direct3D call.

One neat thing about the Evnet List is that if you select a particular draw call, say we picked one of the cube draw calls (notice the index count is 36), the output image on the screen show the objects drawn until that point in time in the event list. Making sure that cube draw call belongs to one of the black cubes and not the walls and floor of the room we’re in (see only 3 black cubes are rendered so far), we can list pretty much all the pipeline stages in detail, can look into every resource that is bound (textures, bufferrs, states etc.).

Let’s look into the index buffer for example.

Index Buffer - click on `obj:11`

Don't forget to change the format to appropriate format to display the buffer


States, Pipeline Stages and Object Table

The middle part of the debug report consists of the current output to the back buffer, and below that are a few nice tabs to show all the related data and pipeline stages.

Render States

This tab lists the resources bound to the selected pipeline stage and values of the render states.


Pipeline Stages

Selecting vertex shader shows the input and the output of the vertex shader for that draw call. It's possible to some transformation errors earlier by just skimming through this table


Object Table

We can see all the Direct3D resourrces, their sizes and their formats. Image shows the depth buffer.


Pixel History

Pixel History

Pixel history can list all the pipeline stage code that worked on a chosen pixel, ready to be debugged on a mouse click.

Shader Debugging

And here’s the killer feature: glorious Visual Studio debugging on the pixel you select drawn at the time you specify during a frame.

Pixel History

Usual Visual Studio debugging, only on shader code. Variables can go out of scope earlier than the end of scope due to the optimizations made by the Direct3D runtime. You can specify the optimization level during device initialization.

If you remember, we’ve set the materials with default constructor on line 21 of SceneManager::InitializeObjectArrays(). Therefore, we were reasonably expecting a 0.0f value for isDiffuseMap. However, the value is actually 1.0f, set by the previous textured object, quite possibly during rendering the floor and walls. Since SetShader() clears all the resources, the pixel shader for the cubes were trying to read from an empty texture unit, hence resulting in black pixels. Let’s review the rendering code:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
void SceneManager::RenderCentralObjects(const XMMATRIX& view, const XMMATRIX& proj) 
{
    // set shader and send constants
    m_renderer->SetShader(m_selectedShader);    //<-- clears all the buffers on GPU when a new shader is set
    if(m_selectedShader == m_renderData->phongShader) SendLightData();
    m_renderer->SetConstant4x4f("view", view);
    m_renderer->SetConstant4x4f("proj", proj);
    m_renderer->SetConstant3f("cameraPos", m_camera->GetPositionF());

    for (const auto& cube : cubes)
    {
        m_renderer->SetBufferObj(cube.m_model.m_mesh);
        //------------------------------------------------ FIX ----------------------------------------------
        cube.m_model.m_material.SetMaterialConstants(m_renderer); // set the material constants for each cube
        //------------------------------------------------ FIX ----------------------------------------------
        XMMATRIX world = cube.m_transform.WorldTransformationMatrix();
        XMMATRIX nrm   = cube.m_transform.NormalMatrix(world);
        m_renderer->SetConstant4x4f("world", world);
        m_renderer->SetConstant4x4f("nrmMatrix", nrm);
        m_renderer->Apply();
        m_renderer->DrawIndexed();
    }
}

And the result is:

Fixed

There we go, that looks like a default material.

Bonus: All default is boring. Lets add some randomness.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
Material Material::RandomMaterial()
{
	float r = RandF(0.0f, 1.0f); // returns [0.0f <= float <= 1.0f]
	if (r >= 0.0000f && r < 0.1999f) return ruby;
	if (r >= 0.2000f && r < 0.3999f) return gold;
	if (r >= 0.4000f && r < 0.5999f) return bronze;
	if (r >= 0.6000f && r < 0.7999f) return jade;
	else return Material();
}

void SceneManager::InitializeObjectArrays()
{
    const int   row = 5,
                col = 5;
    const float r = 3.5f;

    // grid arrangement ( (row * col) cubes that are 'r' apart from each other )
    for (int i = 0; i < row; ++i)
    {
        for (int j = 0; j < col; ++j)
        {
            GameObject cube;

            // set transform
            float x, y, z;  // position
            x = i * r;      y = 5.0f;   z = j * r;
            cube.m_transform.SetPosition(x, y, z);

            // set material
            cube.m_model.m_mesh = MESH_TYPE::CUBE;
            //------------------------------------------------------------------------
            cube.m_model.m_material = RandomMaterial();  // <-- assign random material
            //------------------------------------------------------------------------

            cubes.push_back(cube);
        }
    }
}

Random Colors from preset materials

Random colors from preset materials


Conclusion

As demonstrated above, a decent debugger holds paramount importance in efficient debugging. Utilizing all the system resources (OS, Gfx API & IDE) from a single developer to help debugging – although many people complain about their products constantly – in this case results in a smooth and robust debugging process. Almost none of the GPU debuggers I’ve tried were as stable as Visual Studio or able to expose as much as Visual Studio. Although, I am not a seasoned graphics programmer yet. This means I might be missing out on some of the tools out there. If you happen to know any other decent debugger, feel free to tell me about it’s features on the comment section. I intended this post to be an introductory for DirectX graphics debugging, and hopefully helped with some the basic features of the debugger. There are a lot more to it than what I demonstrated, so I encourage you to read more about it at MSDN. If you happen to have favorite or frequently used feature not mentioned above, let me know about it down below.

Future reading

Written on February 27, 2017