5. Command Buffers

Command buffers are objects used to record commands which can be subsequently submitted to a device queue for execution. There are two levels of command buffers - primary command buffers, which can execute secondary command buffers, and which are submitted to queues, and secondary command buffers, which can be executed by primary command buffers, and which are not directly submitted to queues.

Command buffers are represented by VkCommandBuffer handles:

// Provided by VK_VERSION_1_0
VK_DEFINE_HANDLE(VkCommandBuffer)

Recorded commands include commands to bind pipelines and descriptor sets to the command buffer, commands to modify dynamic state, commands to draw (for graphics rendering), commands to dispatch (for compute), commands to execute secondary command buffers (for primary command buffers only), commands to copy buffers and images, and other commands.

Each command buffer manages state independently of other command buffers. There is no inheritance of state across primary and secondary command buffers, or between secondary command buffers. When a command buffer begins recording, all state in that command buffer is undefined. When secondary command buffer(s) are recorded to execute on a primary command buffer, the secondary command buffer inherits no state from the primary command buffer, and all state of the primary command buffer is undefined after an execute secondary command buffer command is recorded. There is one exception to this rule - if the primary command buffer is inside a render pass instance, then the render pass and subpass state is not disturbed by executing secondary command buffers. For state dependent commands (such as draws and dispatches), any state consumed by those commands must not be undefined.

Unless otherwise specified, and without explicit synchronization, the various commands submitted to a queue via command buffers may execute in arbitrary order relative to each other, and/or concurrently. Also, the memory side-effects of those commands may not be directly visible to other commands without explicit memory dependencies. This is true within a command buffer, and across command buffers submitted to a given queue. See the synchronization chapter for information on implicit and explicit synchronization between commands.

5.1. Command Buffer Lifecycle

Each command buffer is always in one of the following states:

Initial

When a command buffer is allocated, it is in the initial state. Some commands are able to reset a command buffer (or a set of command buffers) back to this state from any of the executable, recording or invalid state. Command buffers in the initial state can only be moved to the recording state, or freed.

Recording

vkBeginCommandBuffer changes the state of a command buffer from the initial state to the recording state. Once a command buffer is in the recording state, vkCmd* commands can be used to record to the command buffer.

Executable

vkEndCommandBuffer ends the recording of a command buffer, and moves it from the recording state to the executable state. Executable command buffers can be submitted, reset, or recorded to another command buffer.

Pending

Queue submission of a command buffer changes the state of a command buffer from the executable state to the pending state. Whilst in the pending state, applications must not attempt to modify the command buffer in any way - as the device may be processing the commands recorded to it. Once execution of a command buffer completes, the command buffer either reverts back to the executable state, or if it was recorded with VK_COMMAND_BUFFER_USAGE_ONE_TIME_SUBMIT_BIT, it moves to the invalid state. A synchronization command should be used to detect when this occurs.

Invalid

Some operations, such as modifying or deleting a resource that was used in a command recorded to a command buffer, will transition the state of that command buffer into the invalid state. Command buffers in the invalid state can only be reset or freed.

image/svg+xml Initial Recording Pending Executable Invalid Allocate Begin End Submission Completion Completion withOne Time Submit Reset Reset Invalidate
Figure 1. Lifecycle of a command buffer

Any given command that operates on a command buffer has its own requirements on what state a command buffer must be in, which are detailed in the valid usage constraints for that command.

Resetting a command buffer is an operation that discards any previously recorded commands and puts a command buffer in the initial state. Resetting occurs as a result of vkResetCommandBuffer or vkResetCommandPool, or as part of vkBeginCommandBuffer (which additionally puts the command buffer in the recording state).

Secondary command buffers can be recorded to a primary command buffer via vkCmdExecuteCommands. This partially ties the lifecycle of the two command buffers together - if the primary is submitted to a queue, both the primary and any secondaries recorded to it move to the pending state. Once execution of the primary completes, so it does for any secondary recorded within it. After all executions of each command buffer complete, they each move to their appropriate completion state (either to the execution state or the invalid state, as specified above).

If a secondary moves to the invalid state or the initial state, then all primary buffers it is recorded in move to the invalid state. A primary moving to any other state does not affect the state of a secondary recorded in it.

Note

Resetting or freeing a primary command buffer removes the lifecycle linkage to all secondary command buffers that were recorded into it.

5.2. Command Pools

Command pools are opaque objects that command buffer memory is allocated from, and which allow the implementation to amortize the cost of resource creation across multiple command buffers. Command pools are externally synchronized, meaning that a command pool must not be used concurrently in multiple threads. That includes use via recording commands on any command buffers allocated from the pool, as well as operations that allocate, free, and reset command buffers or the pool itself.

Command pools are represented by VkCommandPool handles:

// Provided by VK_VERSION_1_0
VK_DEFINE_NON_DISPATCHABLE_HANDLE(VkCommandPool)

To create a command pool, call:

// Provided by VK_VERSION_1_0
VkResult vkCreateCommandPool(
    VkDevice                                    device,
    const VkCommandPoolCreateInfo*              pCreateInfo,
    const VkAllocationCallbacks*                pAllocator,
    VkCommandPool*                              pCommandPool);

  • device is the logical device that creates the command pool.

  • pCreateInfo is a pointer to a VkCommandPoolCreateInfo structure specifying the state of the command pool object.

  • pAllocator controls host memory allocation as described in the Memory Allocation chapter.

  • pCommandPool is a pointer to a VkCommandPool handle in which the created pool is returned.

Valid Usage

  • pCreateInfo->queueFamilyIndex must be the index of a queue family available in the logical device device

Valid Usage (Implicit)
Return Codes
Success

  • VK_SUCCESS

Failure

  • VK_ERROR_OUT_OF_HOST_MEMORY

  • VK_ERROR_OUT_OF_DEVICE_MEMORY

The VkCommandPoolCreateInfo structure is defined as:

// Provided by VK_VERSION_1_0
typedef struct VkCommandPoolCreateInfo {
    VkStructureType             sType;
    const void*                 pNext;
    VkCommandPoolCreateFlags    flags;
    uint32_t                    queueFamilyIndex;
} VkCommandPoolCreateInfo;

  • sType is the type of this structure.

  • pNext is NULL or a pointer to a structure extending this structure.

  • flags is a bitmask of VkCommandPoolCreateFlagBits indicating usage behavior for the pool and command buffers allocated from it.

  • queueFamilyIndex designates a queue family as described in section Queue Family Properties. All command buffers allocated from this command pool must be submitted on queues from the same queue family.

Valid Usage

  • If the protected memory feature is not enabled, the VK_COMMAND_POOL_CREATE_PROTECTED_BIT bit of flags must not be set

Valid Usage (Implicit)

  • sType must be VK_STRUCTURE_TYPE_COMMAND_POOL_CREATE_INFO

  • pNext must be NULL

  • flags must be a valid combination of VkCommandPoolCreateFlagBits values

Bits which can be set in VkCommandPoolCreateInfo::flags to specify usage behavior for a command pool are:

// Provided by VK_VERSION_1_0
typedef enum VkCommandPoolCreateFlagBits {
    VK_COMMAND_POOL_CREATE_TRANSIENT_BIT = 0x00000001,
    VK_COMMAND_POOL_CREATE_RESET_COMMAND_BUFFER_BIT = 0x00000002,
  // Provided by VK_VERSION_1_1
    VK_COMMAND_POOL_CREATE_PROTECTED_BIT = 0x00000004,
} VkCommandPoolCreateFlagBits;

  • VK_COMMAND_POOL_CREATE_TRANSIENT_BIT specifies that command buffers allocated from the pool will be short-lived, meaning that they will be reset or freed in a relatively short timeframe. This flag may be used by the implementation to control memory allocation behavior within the pool.

  • VK_COMMAND_POOL_CREATE_RESET_COMMAND_BUFFER_BIT allows any command buffer allocated from a pool to be individually reset to the initial state; either by calling vkResetCommandBuffer, or via the implicit reset when calling vkBeginCommandBuffer. If this flag is not set on a pool, then vkResetCommandBuffer must not be called for any command buffer allocated from that pool.

  • VK_COMMAND_POOL_CREATE_PROTECTED_BIT specifies that command buffers allocated from the pool are protected command buffers.

// Provided by VK_VERSION_1_0
typedef VkFlags VkCommandPoolCreateFlags;

VkCommandPoolCreateFlags is a bitmask type for setting a mask of zero or more VkCommandPoolCreateFlagBits.

To trim a command pool, call:

// Provided by VK_VERSION_1_1
void vkTrimCommandPool(
    VkDevice                                    device,
    VkCommandPool                               commandPool,
    VkCommandPoolTrimFlags                      flags);

or the equivalent command

// Provided by VK_KHR_maintenance1
void vkTrimCommandPoolKHR(
    VkDevice                                    device,
    VkCommandPool                               commandPool,
    VkCommandPoolTrimFlags                      flags);

  • device is the logical device that owns the command pool.

  • commandPool is the command pool to trim.

  • flags is reserved for future use.

Trimming a command pool recycles unused memory from the command pool back to the system. Command buffers allocated from the pool are not affected by the command.

Note

This command provides applications with some control over the internal memory allocations used by command pools.

Unused memory normally arises from command buffers that have been recorded and later reset, such that they are no longer using the memory. On reset, a command buffer can return memory to its command pool, but the only way to release memory from a command pool to the system requires calling vkResetCommandPool, which cannot be executed while any command buffers from that pool are still in use. Subsequent recording operations into command buffers will re-use this memory but since total memory requirements fluctuate over time, unused memory can accumulate.

In this situation, trimming a command pool may be useful to return unused memory back to the system, returning the total outstanding memory allocated by the pool back to a more “average” value.

Implementations utilize many internal allocation strategies that make it impossible to guarantee that all unused memory is released back to the system. For instance, an implementation of a command pool may involve allocating memory in bulk from the system and sub-allocating from that memory. In such an implementation any live command buffer that holds a reference to a bulk allocation would prevent that allocation from being freed, even if only a small proportion of the bulk allocation is in use.

In most cases trimming will result in a reduction in allocated but unused memory, but it does not guarantee the “ideal” behavior.

Trimming may be an expensive operation, and should not be called frequently. Trimming should be treated as a way to relieve memory pressure after application-known points when there exists enough unused memory that the cost of trimming is “worth” it.
Valid Usage (Implicit)

  • device must be a valid VkDevice handle

  • commandPool must be a valid VkCommandPool handle

  • flags must be 0

  • commandPool must have been created, allocated, or retrieved from device

Host Synchronization

  • Host access to commandPool must be externally synchronized

// Provided by VK_VERSION_1_1
typedef VkFlags VkCommandPoolTrimFlags;

or the equivalent

// Provided by VK_KHR_maintenance1
typedef VkCommandPoolTrimFlags VkCommandPoolTrimFlagsKHR;

VkCommandPoolTrimFlags is a bitmask type for setting a mask, but is currently reserved for future use.

To reset a command pool, call:

// Provided by VK_VERSION_1_0
VkResult vkResetCommandPool(
    VkDevice                                    device,
    VkCommandPool                               commandPool,
    VkCommandPoolResetFlags                     flags);

  • device is the logical device that owns the command pool.

  • commandPool is the command pool to reset.

  • flags is a bitmask of VkCommandPoolResetFlagBits controlling the reset operation.

Resetting a command pool recycles all of the resources from all of the command buffers allocated from the command pool back to the command pool. All command buffers that have been allocated from the command pool are put in the initial state.

Any primary command buffer allocated from another VkCommandPool that is in the recording or executable state and has a secondary command buffer allocated from commandPool recorded into it, becomes invalid.

Valid Usage

  • All VkCommandBuffer objects allocated from commandPool must not be in the pending state

Valid Usage (Implicit)
Host Synchronization

  • Host access to commandPool must be externally synchronized

Return Codes
Success

  • VK_SUCCESS

Failure

  • VK_ERROR_OUT_OF_HOST_MEMORY

  • VK_ERROR_OUT_OF_DEVICE_MEMORY

Bits which can be set in vkResetCommandPool::flags to control the reset operation are:

// Provided by VK_VERSION_1_0
typedef enum VkCommandPoolResetFlagBits {
    VK_COMMAND_POOL_RESET_RELEASE_RESOURCES_BIT = 0x00000001,
} VkCommandPoolResetFlagBits;

  • VK_COMMAND_POOL_RESET_RELEASE_RESOURCES_BIT specifies that resetting a command pool recycles all of the resources from the command pool back to the system.

// Provided by VK_VERSION_1_0
typedef VkFlags VkCommandPoolResetFlags;

VkCommandPoolResetFlags is a bitmask type for setting a mask of zero or more VkCommandPoolResetFlagBits.

To destroy a command pool, call:

// Provided by VK_VERSION_1_0
void vkDestroyCommandPool(
    VkDevice                                    device,
    VkCommandPool                               commandPool,
    const VkAllocationCallbacks*                pAllocator);

  • device is the logical device that destroys the command pool.

  • commandPool is the handle of the command pool to destroy.

  • pAllocator controls host memory allocation as described in the Memory Allocation chapter.

When a pool is destroyed, all command buffers allocated from the pool are freed.

Any primary command buffer allocated from another VkCommandPool that is in the recording or executable state and has a secondary command buffer allocated from commandPool recorded into it, becomes invalid.

Valid Usage

  • All VkCommandBuffer objects allocated from commandPool must not be in the pending state

  • If VkAllocationCallbacks were provided when commandPool was created, a compatible set of callbacks must be provided here

  • If no VkAllocationCallbacks were provided when commandPool was created, pAllocator must be NULL

Valid Usage (Implicit)

  • device must be a valid VkDevice handle

  • If commandPool is not VK_NULL_HANDLE, commandPool must be a valid VkCommandPool handle

  • If pAllocator is not NULL, pAllocator must be a valid pointer to a valid VkAllocationCallbacks structure

  • If commandPool is a valid handle, it must have been created, allocated, or retrieved from device

Host Synchronization

  • Host access to commandPool must be externally synchronized

5.3. Command Buffer Allocation and Management

To allocate command buffers, call:

// Provided by VK_VERSION_1_0
VkResult vkAllocateCommandBuffers(
    VkDevice                                    device,
    const VkCommandBufferAllocateInfo*          pAllocateInfo,
    VkCommandBuffer*                            pCommandBuffers);

  • device is the logical device that owns the command pool.

  • pAllocateInfo is a pointer to a VkCommandBufferAllocateInfo structure describing parameters of the allocation.

  • pCommandBuffers is a pointer to an array of VkCommandBuffer handles in which the resulting command buffer objects are returned. The array must be at least the length specified by the commandBufferCount member of pAllocateInfo. Each allocated command buffer begins in the initial state.

vkAllocateCommandBuffers can be used to create multiple command buffers. If the creation of any of those command buffers fails, the implementation must destroy all successfully created command buffer objects from this command, set all entries of the pCommandBuffers array to NULL and return the error.

When command buffers are first allocated, they are in the initial state.

Valid Usage (Implicit)

  • device must be a valid VkDevice handle

  • pAllocateInfo must be a valid pointer to a valid VkCommandBufferAllocateInfo structure

  • pCommandBuffers must be a valid pointer to an array of pAllocateInfo->commandBufferCount VkCommandBuffer handles

  • pAllocateInfo->commandBufferCount must be greater than 0

Host Synchronization

  • Host access to pAllocateInfo->commandPool must be externally synchronized

Return Codes
Success

  • VK_SUCCESS

Failure

  • VK_ERROR_OUT_OF_HOST_MEMORY

  • VK_ERROR_OUT_OF_DEVICE_MEMORY

The VkCommandBufferAllocateInfo structure is defined as:

// Provided by VK_VERSION_1_0
typedef struct VkCommandBufferAllocateInfo {
    VkStructureType         sType;
    const void*             pNext;
    VkCommandPool           commandPool;
    VkCommandBufferLevel    level;
    uint32_t                commandBufferCount;
} VkCommandBufferAllocateInfo;

  • sType is the type of this structure.

  • pNext is NULL or a pointer to a structure extending this structure.

  • commandPool is the command pool from which the command buffers are allocated.

  • level is a VkCommandBufferLevel value specifying the command buffer level.

  • commandBufferCount is the number of command buffers to allocate from the pool.

Valid Usage

  • commandBufferCount must be greater than 0

Valid Usage (Implicit)

  • sType must be VK_STRUCTURE_TYPE_COMMAND_BUFFER_ALLOCATE_INFO

  • pNext must be NULL

  • commandPool must be a valid VkCommandPool handle

  • level must be a valid VkCommandBufferLevel value

Possible values of VkCommandBufferAllocateInfo::level, specifying the command buffer level, are:

// Provided by VK_VERSION_1_0
typedef enum VkCommandBufferLevel {
    VK_COMMAND_BUFFER_LEVEL_PRIMARY = 0,
    VK_COMMAND_BUFFER_LEVEL_SECONDARY = 1,
} VkCommandBufferLevel;

  • VK_COMMAND_BUFFER_LEVEL_PRIMARY specifies a primary command buffer.

  • VK_COMMAND_BUFFER_LEVEL_SECONDARY specifies a secondary command buffer.

To reset command buffers, call:

// Provided by VK_VERSION_1_0
VkResult vkResetCommandBuffer(
    VkCommandBuffer                             commandBuffer,
    VkCommandBufferResetFlags                   flags);

Any primary command buffer that is in the recording or executable state and has commandBuffer recorded into it, becomes invalid.

Valid Usage

  • commandBuffer must not be in the pending state

  • commandBuffer must have been allocated from a pool that was created with the VK_COMMAND_POOL_CREATE_RESET_COMMAND_BUFFER_BIT

Valid Usage (Implicit)
Host Synchronization

  • Host access to commandBuffer must be externally synchronized

Return Codes
Success

  • VK_SUCCESS

Failure

  • VK_ERROR_OUT_OF_HOST_MEMORY

  • VK_ERROR_OUT_OF_DEVICE_MEMORY

Bits which can be set in vkResetCommandBuffer::flags to control the reset operation are:

// Provided by VK_VERSION_1_0
typedef enum VkCommandBufferResetFlagBits {
    VK_COMMAND_BUFFER_RESET_RELEASE_RESOURCES_BIT = 0x00000001,
} VkCommandBufferResetFlagBits;

  • VK_COMMAND_BUFFER_RESET_RELEASE_RESOURCES_BIT specifies that most or all memory resources currently owned by the command buffer should be returned to the parent command pool. If this flag is not set, then the command buffer may hold onto memory resources and reuse them when recording commands. commandBuffer is moved to the initial state.

// Provided by VK_VERSION_1_0
typedef VkFlags VkCommandBufferResetFlags;

VkCommandBufferResetFlags is a bitmask type for setting a mask of zero or more VkCommandBufferResetFlagBits.

To free command buffers, call:

// Provided by VK_VERSION_1_0
void vkFreeCommandBuffers(
    VkDevice                                    device,
    VkCommandPool                               commandPool,
    uint32_t                                    commandBufferCount,
    const VkCommandBuffer*                      pCommandBuffers);

  • device is the logical device that owns the command pool.

  • commandPool is the command pool from which the command buffers were allocated.

  • commandBufferCount is the length of the pCommandBuffers array.

  • pCommandBuffers is a pointer to an array of handles of command buffers to free.

Any primary command buffer that is in the recording or executable state and has any element of pCommandBuffers recorded into it, becomes invalid.

Valid Usage

  • All elements of pCommandBuffers must not be in the pending state

  • pCommandBuffers must be a valid pointer to an array of commandBufferCount VkCommandBuffer handles, each element of which must either be a valid handle or NULL

Valid Usage (Implicit)

  • device must be a valid VkDevice handle

  • commandPool must be a valid VkCommandPool handle

  • commandBufferCount must be greater than 0

  • commandPool must have been created, allocated, or retrieved from device

  • Each element of pCommandBuffers that is a valid handle must have been created, allocated, or retrieved from commandPool

Host Synchronization

  • Host access to commandPool must be externally synchronized

  • Host access to each member of pCommandBuffers must be externally synchronized

5.4. Command Buffer Recording

To begin recording a command buffer, call:

// Provided by VK_VERSION_1_0
VkResult vkBeginCommandBuffer(
    VkCommandBuffer                             commandBuffer,
    const VkCommandBufferBeginInfo*             pBeginInfo);

  • commandBuffer is the handle of the command buffer which is to be put in the recording state.

  • pBeginInfo points to a VkCommandBufferBeginInfo structure defining additional information about how the command buffer begins recording.

Valid Usage

  • commandBuffer must not be in the recording or pending state

  • If commandBuffer was allocated from a VkCommandPool which did not have the VK_COMMAND_POOL_CREATE_RESET_COMMAND_BUFFER_BIT flag set, commandBuffer must be in the initial state

  • If commandBuffer is a secondary command buffer, the pInheritanceInfo member of pBeginInfo must be a valid VkCommandBufferInheritanceInfo structure

  • If commandBuffer is a secondary command buffer and either the occlusionQueryEnable member of the pInheritanceInfo member of pBeginInfo is VK_FALSE, or the precise occlusion queries feature is not enabled, the queryFlags member of the pInheritanceInfo member pBeginInfo must not contain VK_QUERY_CONTROL_PRECISE_BIT

  • If commandBuffer is a primary command buffer, then pBeginInfo->flags must not set both the VK_COMMAND_BUFFER_USAGE_ONE_TIME_SUBMIT_BIT and the VK_COMMAND_BUFFER_USAGE_SIMULTANEOUS_USE_BIT flags

Valid Usage (Implicit)
Host Synchronization

  • Host access to commandBuffer must be externally synchronized

  • Host access to the VkCommandPool that commandBuffer was allocated from must be externally synchronized

Return Codes
Success

  • VK_SUCCESS

Failure

  • VK_ERROR_OUT_OF_HOST_MEMORY

  • VK_ERROR_OUT_OF_DEVICE_MEMORY

The VkCommandBufferBeginInfo structure is defined as:

// Provided by VK_VERSION_1_0
typedef struct VkCommandBufferBeginInfo {
    VkStructureType                          sType;
    const void*                              pNext;
    VkCommandBufferUsageFlags                flags;
    const VkCommandBufferInheritanceInfo*    pInheritanceInfo;
} VkCommandBufferBeginInfo;

  • sType is the type of this structure.

  • pNext is NULL or a pointer to a structure extending this structure.

  • flags is a bitmask of VkCommandBufferUsageFlagBits specifying usage behavior for the command buffer.

  • pInheritanceInfo is a pointer to a VkCommandBufferInheritanceInfo structure, used if commandBuffer is a secondary command buffer. If this is a primary command buffer, then this value is ignored.

Valid Usage

  • If flags contains VK_COMMAND_BUFFER_USAGE_RENDER_PASS_CONTINUE_BIT, the renderPass member of pInheritanceInfo must be a valid VkRenderPass

  • If flags contains VK_COMMAND_BUFFER_USAGE_RENDER_PASS_CONTINUE_BIT, the subpass member of pInheritanceInfo must be a valid subpass index within the renderPass member of pInheritanceInfo

  • If flags contains VK_COMMAND_BUFFER_USAGE_RENDER_PASS_CONTINUE_BIT, the framebuffer member of pInheritanceInfo must be either VK_NULL_HANDLE, or a valid VkFramebuffer that is compatible with the renderPass member of pInheritanceInfo

Valid Usage (Implicit)

Bits which can be set in VkCommandBufferBeginInfo::flags to specify usage behavior for a command buffer are:

// Provided by VK_VERSION_1_0
typedef enum VkCommandBufferUsageFlagBits {
    VK_COMMAND_BUFFER_USAGE_ONE_TIME_SUBMIT_BIT = 0x00000001,
    VK_COMMAND_BUFFER_USAGE_RENDER_PASS_CONTINUE_BIT = 0x00000002,
    VK_COMMAND_BUFFER_USAGE_SIMULTANEOUS_USE_BIT = 0x00000004,
} VkCommandBufferUsageFlagBits;

  • VK_COMMAND_BUFFER_USAGE_ONE_TIME_SUBMIT_BIT specifies that each recording of the command buffer will only be submitted once, and the command buffer will be reset and recorded again between each submission.

  • VK_COMMAND_BUFFER_USAGE_RENDER_PASS_CONTINUE_BIT specifies that a secondary command buffer is considered to be entirely inside a render pass. If this is a primary command buffer, then this bit is ignored.

  • VK_COMMAND_BUFFER_USAGE_SIMULTANEOUS_USE_BIT specifies that a command buffer can be resubmitted to a queue while it is in the pending state, and recorded into multiple primary command buffers.

// Provided by VK_VERSION_1_0
typedef VkFlags VkCommandBufferUsageFlags;

VkCommandBufferUsageFlags is a bitmask type for setting a mask of zero or more VkCommandBufferUsageFlagBits.

If the command buffer is a secondary command buffer, then the VkCommandBufferInheritanceInfo structure defines any state that will be inherited from the primary command buffer:

// Provided by VK_VERSION_1_0
typedef struct VkCommandBufferInheritanceInfo {
    VkStructureType                  sType;
    const void*                      pNext;
    VkRenderPass                     renderPass;
    uint32_t                         subpass;
    VkFramebuffer                    framebuffer;
    VkBool32                         occlusionQueryEnable;
    VkQueryControlFlags              queryFlags;
    VkQueryPipelineStatisticFlags    pipelineStatistics;
} VkCommandBufferInheritanceInfo;

  • sType is the type of this structure.

  • pNext is NULL or a pointer to a structure extending this structure.

  • renderPass is a VkRenderPass object defining which render passes the VkCommandBuffer will be compatible with and can be executed within. If the VkCommandBuffer will not be executed within a render pass instance, renderPass is ignored.

  • subpass is the index of the subpass within the render pass instance that the VkCommandBuffer will be executed within. If the VkCommandBuffer will not be executed within a render pass instance, subpass is ignored.

  • framebuffer optionally refers to the VkFramebuffer object that the VkCommandBuffer will be rendering to if it is executed within a render pass instance. It can be VK_NULL_HANDLE if the framebuffer is not known, or if the VkCommandBuffer will not be executed within a render pass instance.

    Note

    Specifying the exact framebuffer that the secondary command buffer will be executed with may result in better performance at command buffer execution time.

  • occlusionQueryEnable specifies whether the command buffer can be executed while an occlusion query is active in the primary command buffer. If this is VK_TRUE, then this command buffer can be executed whether the primary command buffer has an occlusion query active or not. If this is VK_FALSE, then the primary command buffer must not have an occlusion query active.

  • queryFlags specifies the query flags that can be used by an active occlusion query in the primary command buffer when this secondary command buffer is executed. If this value includes the VK_QUERY_CONTROL_PRECISE_BIT bit, then the active query can return boolean results or actual sample counts. If this bit is not set, then the active query must not use the VK_QUERY_CONTROL_PRECISE_BIT bit.

  • pipelineStatistics is a bitmask of VkQueryPipelineStatisticFlagBits specifying the set of pipeline statistics that can be counted by an active query in the primary command buffer when this secondary command buffer is executed. If this value includes a given bit, then this command buffer can be executed whether the primary command buffer has a pipeline statistics query active that includes this bit or not. If this value excludes a given bit, then the active pipeline statistics query must not be from a query pool that counts that statistic.

Valid Usage
Valid Usage (Implicit)
Note

On some implementations, not using the VK_COMMAND_BUFFER_USAGE_SIMULTANEOUS_USE_BIT bit enables command buffers to be patched in-place if needed, rather than creating a copy of the command buffer.

If a command buffer is in the invalid, or executable state, and the command buffer was allocated from a command pool with the VK_COMMAND_POOL_CREATE_RESET_COMMAND_BUFFER_BIT flag set, then vkBeginCommandBuffer implicitly resets the command buffer, behaving as if vkResetCommandBuffer had been called with VK_COMMAND_BUFFER_RESET_RELEASE_RESOURCES_BIT not set. After the implicit reset, commandBuffer is moved to the recording state.

If the pNext chain of VkCommandBufferInheritanceInfo includes a VkCommandBufferInheritanceConditionalRenderingInfoEXT structure, then that structure controls whether a command buffer can be executed while conditional rendering is active in the primary command buffer.

The VkCommandBufferInheritanceConditionalRenderingInfoEXT structure is defined as:

// Provided by VK_EXT_conditional_rendering
typedef struct VkCommandBufferInheritanceConditionalRenderingInfoEXT {
    VkStructureType    sType;
    const void*        pNext;
    VkBool32           conditionalRenderingEnable;
} VkCommandBufferInheritanceConditionalRenderingInfoEXT;

  • sType is the type of this structure

  • pNext is NULL or a pointer to a structure extending this structure

  • conditionalRenderingEnable specifies whether the command buffer can be executed while conditional rendering is active in the primary command buffer. If this is VK_TRUE, then this command buffer can be executed whether the primary command buffer has active conditional rendering or not. If this is VK_FALSE, then the primary command buffer must not have conditional rendering active.

If this structure is not present, the behavior is as if conditionalRenderingEnable is VK_FALSE.

Valid Usage
Valid Usage (Implicit)

  • sType must be VK_STRUCTURE_TYPE_COMMAND_BUFFER_INHERITANCE_CONDITIONAL_RENDERING_INFO_EXT

To begin recording a secondary command buffer compatible with execution inside a render pass using render pass transform, add the VkCommandBufferInheritanceRenderPassTransformInfoQCOM to the pNext chain of VkCommandBufferInheritanceInfo structure passed to the vkBeginCommandBuffer command specifying the parameters for transformed rasterization.

The VkCommandBufferInheritanceRenderPassTransformInfoQCOM structure is defined as:

// Provided by VK_QCOM_render_pass_transform
typedef struct VkCommandBufferInheritanceRenderPassTransformInfoQCOM {
    VkStructureType                  sType;
    void*                            pNext;
    VkSurfaceTransformFlagBitsKHR    transform;
    VkRect2D                         renderArea;
} VkCommandBufferInheritanceRenderPassTransformInfoQCOM;

  • sType is the type of this structure.

  • pNext is NULL or a pointer to a structure extending this structure.

  • transform is a VkSurfaceTransformFlagBitsKHR value describing the transform to be applied to the render pass.

  • renderArea is the render area that is affected by the command buffer.

When the secondary is recorded to execute within a render pass instance using vkCmdExecuteCommands, the render pass transform parameters of the secondary command buffer must be consistent with the render pass transform parameters specified for the render pass instance. In particular, the transform and renderArea for command buffer must be identical to the transform and renderArea of the render pass instance.

Valid Usage

  • transform must be VK_SURFACE_TRANSFORM_IDENTITY_BIT_KHR, VK_SURFACE_TRANSFORM_ROTATE_90_BIT_KHR, VK_SURFACE_TRANSFORM_ROTATE_180_BIT_KHR, or VK_SURFACE_TRANSFORM_ROTATE_270_BIT_KHR

Valid Usage (Implicit)

  • sType must be VK_STRUCTURE_TYPE_COMMAND_BUFFER_INHERITANCE_RENDER_PASS_TRANSFORM_INFO_QCOM

Once recording starts, an application records a sequence of commands (vkCmd*) to set state in the command buffer, draw, dispatch, and other commands.

Several commands can also be recorded indirectly from VkBuffer content, see Device-Generated Commands.

To complete recording of a command buffer, call:

// Provided by VK_VERSION_1_0
VkResult vkEndCommandBuffer(
    VkCommandBuffer                             commandBuffer);

  • commandBuffer is the command buffer to complete recording.

If there was an error during recording, the application will be notified by an unsuccessful return code returned by vkEndCommandBuffer. If the application wishes to further use the command buffer, the command buffer must be reset. The command buffer must have been in the recording state, and is moved to the executable state.

Valid Usage

  • commandBuffer must be in the recording state

  • If commandBuffer is a primary command buffer, there must not be an active render pass instance

  • All queries made active during the recording of commandBuffer must have been made inactive

  • Conditional rendering must not be active

  • If commandBuffer is a secondary command buffer, there must not be an outstanding vkCmdBeginDebugUtilsLabelEXT command recorded to commandBuffer that has not previously been ended by a call to vkCmdEndDebugUtilsLabelEXT

  • If commandBuffer is a secondary command buffer, there must not be an outstanding vkCmdDebugMarkerBeginEXT command recorded to commandBuffer that has not previously been ended by a call to vkCmdDebugMarkerEndEXT

Valid Usage (Implicit)
Host Synchronization

  • Host access to commandBuffer must be externally synchronized

  • Host access to the VkCommandPool that commandBuffer was allocated from must be externally synchronized

Return Codes
Success

  • VK_SUCCESS

Failure

  • VK_ERROR_OUT_OF_HOST_MEMORY

  • VK_ERROR_OUT_OF_DEVICE_MEMORY

When a command buffer is in the executable state, it can be submitted to a queue for execution.

5.5. Command Buffer Submission

Note

Submission can be a high overhead operation, and applications should attempt to batch work together into as few calls to vkQueueSubmit as possible.

To submit command buffers to a queue, call:

// Provided by VK_VERSION_1_0
VkResult vkQueueSubmit(
    VkQueue                                     queue,
    uint32_t                                    submitCount,
    const VkSubmitInfo*                         pSubmits,
    VkFence                                     fence);

  • queue is the queue that the command buffers will be submitted to.

  • submitCount is the number of elements in the pSubmits array.

  • pSubmits is a pointer to an array of VkSubmitInfo structures, each specifying a command buffer submission batch.

  • fence is an optional handle to a fence to be signaled once all submitted command buffers have completed execution. If fence is not VK_NULL_HANDLE, it defines a fence signal operation.

vkQueueSubmit is a queue submission command, with each batch defined by an element of pSubmits. Batches begin execution in the order they appear in pSubmits, but may complete out of order.

Fence and semaphore operations submitted with vkQueueSubmit have additional ordering constraints compared to other submission commands, with dependencies involving previous and subsequent queue operations. Information about these additional constraints can be found in the semaphore and fence sections of the synchronization chapter.

Details on the interaction of pWaitDstStageMask with synchronization are described in the semaphore wait operation section of the synchronization chapter.

The order that batches appear in pSubmits is used to determine submission order, and thus all the implicit ordering guarantees that respect it. Other than these implicit ordering guarantees and any explicit synchronization primitives, these batches may overlap or otherwise execute out of order.

If any command buffer submitted to this queue is in the executable state, it is moved to the pending state. Once execution of all submissions of a command buffer complete, it moves from the pending state, back to the executable state. If a command buffer was recorded with the VK_COMMAND_BUFFER_USAGE_ONE_TIME_SUBMIT_BIT flag, it instead moves to the invalid state.

If vkQueueSubmit fails, it may return VK_ERROR_OUT_OF_HOST_MEMORY or VK_ERROR_OUT_OF_DEVICE_MEMORY. If it does, the implementation must ensure that the state and contents of any resources or synchronization primitives referenced by the submitted command buffers and any semaphores referenced by pSubmits is unaffected by the call or its failure. If vkQueueSubmit fails in such a way that the implementation is unable to make that guarantee, the implementation must return VK_ERROR_DEVICE_LOST. See Lost Device.

Valid Usage

  • If fence is not VK_NULL_HANDLE, fence must be unsignaled

  • If fence is not VK_NULL_HANDLE, fence must not be associated with any other queue command that has not yet completed execution on that queue

  • Any calls to vkCmdSetEvent, vkCmdResetEvent or vkCmdWaitEvents that have been recorded into any of the command buffer elements of the pCommandBuffers member of any element of pSubmits, must not reference any VkEvent that is referenced by any of those commands in a command buffer that has been submitted to another queue and is still in the pending state

  • Any stage flag included in any element of the pWaitDstStageMask member of any element of pSubmits must be a pipeline stage supported by one of the capabilities of queue, as specified in the table of supported pipeline stages

  • Each element of the pSignalSemaphores member of any element of pSubmits must be unsignaled when the semaphore signal operation it defines is executed on the device

  • When a semaphore wait operation referring to a binary semaphore defined by any element of the pWaitSemaphores member of any element of pSubmits executes on queue, there must be no other queues waiting on the same semaphore

  • All elements of the pWaitSemaphores member of all elements of pSubmits created with a VkSemaphoreType of VK_SEMAPHORE_TYPE_BINARY must reference a semaphore signal operation that has been submitted for execution and any semaphore signal operations on which it depends (if any) must have also been submitted for execution

  • Each element of the pCommandBuffers member of each element of pSubmits must be in the pending or executable state

  • If any element of the pCommandBuffers member of any element of pSubmits was not recorded with the VK_COMMAND_BUFFER_USAGE_SIMULTANEOUS_USE_BIT, it must not be in the pending state

  • Any secondary command buffers recorded into any element of the pCommandBuffers member of any element of pSubmits must be in the pending or executable state

  • If any secondary command buffers recorded into any element of the pCommandBuffers member of any element of pSubmits was not recorded with the VK_COMMAND_BUFFER_USAGE_SIMULTANEOUS_USE_BIT, it must not be in the pending state

  • Each element of the pCommandBuffers member of each element of pSubmits must have been allocated from a VkCommandPool that was created for the same queue family queue belongs to

  • If any element of pSubmits->pCommandBuffers includes a Queue Family Transfer Acquire Operation, there must exist a previously submitted Queue Family Transfer Release Operation on a queue in the queue family identified by the acquire operation, with parameters matching the acquire operation as defined in the definition of such acquire operations, and which happens-before the acquire operation

  • If a command recorded into any element of pCommandBuffers was a vkCmdBeginQuery whose queryPool was created with a queryType of VK_QUERY_TYPE_PERFORMANCE_QUERY_KHR, the profiling lock must have been held continuously on the VkDevice that queue was retrieved from, throughout recording of those command buffers

  • Any resource created with VK_SHARING_MODE_EXCLUSIVE that is read by an operation specified by pSubmits must not be owned by any queue family other than the one which queue belongs to, at the time it is executed

Valid Usage (Implicit)

  • queue must be a valid VkQueue handle

  • If submitCount is not 0, pSubmits must be a valid pointer to an array of submitCount valid VkSubmitInfo structures

  • If fence is not VK_NULL_HANDLE, fence must be a valid VkFence handle

  • Both of fence, and queue that are valid handles of non-ignored parameters must have been created, allocated, or retrieved from the same VkDevice

Host Synchronization

  • Host access to queue must be externally synchronized

  • Host access to fence must be externally synchronized

Command Properties
Command Buffer Levels Render Pass Scope Supported Queue Types Pipeline Type

-

-

Any

-
Return Codes
Success

  • VK_SUCCESS

Failure

  • VK_ERROR_OUT_OF_HOST_MEMORY

  • VK_ERROR_OUT_OF_DEVICE_MEMORY

  • VK_ERROR_DEVICE_LOST

The VkSubmitInfo structure is defined as:

// Provided by VK_VERSION_1_0
typedef struct VkSubmitInfo {
    VkStructureType                sType;
    const void*                    pNext;
    uint32_t                       waitSemaphoreCount;
    const VkSemaphore*             pWaitSemaphores;
    const VkPipelineStageFlags*    pWaitDstStageMask;
    uint32_t                       commandBufferCount;
    const VkCommandBuffer*         pCommandBuffers;
    uint32_t                       signalSemaphoreCount;
    const VkSemaphore*             pSignalSemaphores;
} VkSubmitInfo;

  • sType is the type of this structure.

  • pNext is NULL or a pointer to a structure extending this structure.

  • waitSemaphoreCount is the number of semaphores upon which to wait before executing the command buffers for the batch.

  • pWaitSemaphores is a pointer to an array of VkSemaphore handles upon which to wait before the command buffers for this batch begin execution. If semaphores to wait on are provided, they define a semaphore wait operation.

  • pWaitDstStageMask is a pointer to an array of pipeline stages at which each corresponding semaphore wait will occur.

  • commandBufferCount is the number of command buffers to execute in the batch.

  • pCommandBuffers is a pointer to an array of VkCommandBuffer handles to execute in the batch.

  • signalSemaphoreCount is the number of semaphores to be signaled once the commands specified in pCommandBuffers have completed execution.

  • pSignalSemaphores is a pointer to an array of VkSemaphore handles which will be signaled when the command buffers for this batch have completed execution. If semaphores to be signaled are provided, they define a semaphore signal operation.

The order that command buffers appear in pCommandBuffers is used to determine submission order, and thus all the implicit ordering guarantees that respect it. Other than these implicit ordering guarantees and any explicit synchronization primitives, these command buffers may overlap or otherwise execute out of order.

Valid Usage

  • Each element of pCommandBuffers must not have been allocated with VK_COMMAND_BUFFER_LEVEL_SECONDARY

  • If the geometry shaders feature is not enabled, each element of pWaitDstStageMask must not contain VK_PIPELINE_STAGE_GEOMETRY_SHADER_BIT

  • If the tessellation shaders feature is not enabled, each element of pWaitDstStageMask must not contain VK_PIPELINE_STAGE_TESSELLATION_CONTROL_SHADER_BIT or VK_PIPELINE_STAGE_TESSELLATION_EVALUATION_SHADER_BIT

  • Each element of pWaitDstStageMask must not include VK_PIPELINE_STAGE_HOST_BIT

  • If any element of pWaitSemaphores or pSignalSemaphores was created with a VkSemaphoreType of VK_SEMAPHORE_TYPE_TIMELINE, then the pNext chain must include a VkTimelineSemaphoreSubmitInfo structure

  • If the pNext chain of this structure includes a VkTimelineSemaphoreSubmitInfo structure and any element of pWaitSemaphores was created with a VkSemaphoreType of VK_SEMAPHORE_TYPE_TIMELINE, then its waitSemaphoreValueCount member must equal waitSemaphoreCount

  • If the pNext chain of this structure includes a VkTimelineSemaphoreSubmitInfo structure and any element of pSignalSemaphores was created with a VkSemaphoreType of VK_SEMAPHORE_TYPE_TIMELINE, then its signalSemaphoreValueCount member must equal signalSemaphoreCount

  • For each element of pSignalSemaphores created with a VkSemaphoreType of VK_SEMAPHORE_TYPE_TIMELINE the corresponding element of VkTimelineSemaphoreSubmitInfo::pSignalSemaphoreValues must have a value greater than the current value of the semaphore when the semaphore signal operation is executed

  • For each element of pWaitSemaphores created with a VkSemaphoreType of VK_SEMAPHORE_TYPE_TIMELINE the corresponding element of VkTimelineSemaphoreSubmitInfo::pWaitSemaphoreValues must have a value which does not differ from the current value of the semaphore or the value of any outstanding semaphore wait or signal operation on that semaphore by more than maxTimelineSemaphoreValueDifference

  • For each element of pSignalSemaphores created with a VkSemaphoreType of VK_SEMAPHORE_TYPE_TIMELINE the corresponding element of VkTimelineSemaphoreSubmitInfo::pSignalSemaphoreValues must have a value which does not differ from the current value of the semaphore or the value of any outstanding semaphore wait or signal operation on that semaphore by more than maxTimelineSemaphoreValueDifference

  • If the mesh shaders feature is not enabled, each element of pWaitDstStageMask must not contain VK_PIPELINE_STAGE_MESH_SHADER_BIT_NV

  • If the task shaders feature is not enabled, each element of pWaitDstStageMask must not contain VK_PIPELINE_STAGE_TASK_SHADER_BIT_NV

  • If the pNext chain of this structure does not include a VkProtectedSubmitInfo structure with protectedSubmit set to VK_TRUE, then each element of the command buffer of the pCommandBuffers array must be an unprotected command buffer

Valid Usage (Implicit)

To specify the values to use when waiting for and signaling semaphores created with a VkSemaphoreType of VK_SEMAPHORE_TYPE_TIMELINE, add a VkTimelineSemaphoreSubmitInfo structure to the pNext chain of the VkSubmitInfo structure when using vkQueueSubmit or the VkBindSparseInfo structure when using vkQueueBindSparse. The VkTimelineSemaphoreSubmitInfo structure is defined as:

// Provided by VK_VERSION_1_2
typedef struct VkTimelineSemaphoreSubmitInfo {
    VkStructureType    sType;
    const void*        pNext;
    uint32_t           waitSemaphoreValueCount;
    const uint64_t*    pWaitSemaphoreValues;
    uint32_t           signalSemaphoreValueCount;
    const uint64_t*    pSignalSemaphoreValues;
} VkTimelineSemaphoreSubmitInfo;

or the equivalent

// Provided by VK_KHR_timeline_semaphore
typedef VkTimelineSemaphoreSubmitInfo VkTimelineSemaphoreSubmitInfoKHR;

  • sType is the type of this structure.

  • pNext is NULL or a pointer to a structure extending this structure.

  • waitSemaphoreValueCount is the number of semaphore wait values specified in pWaitSemaphoreValues.

  • pWaitSemaphoreValues is an array of length waitSemaphoreValueCount containing values for the corresponding semaphores in VkSubmitInfo::pWaitSemaphores to wait for.

  • signalSemaphoreValueCount is the number of semaphore signal values specified in pSignalSemaphoreValues.

  • pSignalSemaphoreValues is an array of length signalSemaphoreValueCount containing values for the corresponding semaphores in VkSubmitInfo::pSignalSemaphores to set when signaled.

If the semaphore in VkSubmitInfo::pWaitSemaphores or VkSubmitInfo::pSignalSemaphores corresponding to an entry in pWaitSemaphoreValues or pSignalSemaphoreValues respectively was not created with a VkSemaphoreType of VK_SEMAPHORE_TYPE_TIMELINE, the implementation must ignore the value in the pWaitSemaphoreValues or pSignalSemaphoreValues entry.

Valid Usage (Implicit)

  • sType must be VK_STRUCTURE_TYPE_TIMELINE_SEMAPHORE_SUBMIT_INFO

  • If waitSemaphoreValueCount is not 0, and pWaitSemaphoreValues is not NULL, pWaitSemaphoreValues must be a valid pointer to an array of waitSemaphoreValueCount uint64_t values

  • If signalSemaphoreValueCount is not 0, and pSignalSemaphoreValues is not NULL, pSignalSemaphoreValues must be a valid pointer to an array of signalSemaphoreValueCount uint64_t values

To specify the values to use when waiting for and signaling semaphores whose current payload refers to a Direct3D 12 fence, add a VkD3D12FenceSubmitInfoKHR structure to the pNext chain of the VkSubmitInfo structure. The VkD3D12FenceSubmitInfoKHR structure is defined as:

// Provided by VK_KHR_external_semaphore_win32
typedef struct VkD3D12FenceSubmitInfoKHR {
    VkStructureType    sType;
    const void*        pNext;
    uint32_t           waitSemaphoreValuesCount;
    const uint64_t*    pWaitSemaphoreValues;
    uint32_t           signalSemaphoreValuesCount;
    const uint64_t*    pSignalSemaphoreValues;
} VkD3D12FenceSubmitInfoKHR;

  • sType is the type of this structure.

  • pNext is NULL or a pointer to a structure extending this structure.

  • waitSemaphoreValuesCount is the number of semaphore wait values specified in pWaitSemaphoreValues.

  • pWaitSemaphoreValues is a pointer to an array of waitSemaphoreValuesCount values for the corresponding semaphores in VkSubmitInfo::pWaitSemaphores to wait for.

  • signalSemaphoreValuesCount is the number of semaphore signal values specified in pSignalSemaphoreValues.

  • pSignalSemaphoreValues is a pointer to an array of signalSemaphoreValuesCount values for the corresponding semaphores in VkSubmitInfo::pSignalSemaphores to set when signaled.

If the semaphore in VkSubmitInfo::pWaitSemaphores or VkSubmitInfo::pSignalSemaphores corresponding to an entry in pWaitSemaphoreValues or pSignalSemaphoreValues respectively does not currently have a payload referring to a Direct3D 12 fence, the implementation must ignore the value in the pWaitSemaphoreValues or pSignalSemaphoreValues entry.

Note

As the introduction of the external semaphore handle type VK_EXTERNAL_SEMAPHORE_HANDLE_TYPE_D3D12_FENCE_BIT predates that of timeline semaphores, support for importing semaphore payloads from external handles of that type into semaphores created (implicitly or explicitly) with a VkSemaphoreType of VK_SEMAPHORE_TYPE_BINARY is preserved for backwards compatibility. However, applications should prefer importing such handle types into semaphores created with a VkSemaphoreType of VK_SEMAPHORE_TYPE_TIMELINE, and use the VkTimelineSemaphoreSubmitInfo structure instead of the VkD3D12FenceSubmitInfoKHR structure to specify the values to use when waiting for and signaling such semaphores.
Valid Usage

  • waitSemaphoreValuesCount must be the same value as VkSubmitInfo::waitSemaphoreCount, where VkSubmitInfo is in the pNext chain of this VkD3D12FenceSubmitInfoKHR structure

  • signalSemaphoreValuesCount must be the same value as VkSubmitInfo::signalSemaphoreCount, where VkSubmitInfo is in the pNext chain of this VkD3D12FenceSubmitInfoKHR structure

Valid Usage (Implicit)

  • sType must be VK_STRUCTURE_TYPE_D3D12_FENCE_SUBMIT_INFO_KHR

  • If waitSemaphoreValuesCount is not 0, and pWaitSemaphoreValues is not NULL, pWaitSemaphoreValues must be a valid pointer to an array of waitSemaphoreValuesCount uint64_t values

  • If signalSemaphoreValuesCount is not 0, and pSignalSemaphoreValues is not NULL, pSignalSemaphoreValues must be a valid pointer to an array of signalSemaphoreValuesCount uint64_t values

When submitting work that operates on memory imported from a Direct3D 11 resource to a queue, the keyed mutex mechanism may be used in addition to Vulkan semaphores to synchronize the work. Keyed mutexes are a property of a properly created shareable Direct3D 11 resource. They can only be used if the imported resource was created with the D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX flag.

To acquire keyed mutexes before submitted work and/or release them after, add a VkWin32KeyedMutexAcquireReleaseInfoKHR structure to the pNext chain of the VkSubmitInfo structure.

The VkWin32KeyedMutexAcquireReleaseInfoKHR structure is defined as:

// Provided by VK_KHR_win32_keyed_mutex
typedef struct VkWin32KeyedMutexAcquireReleaseInfoKHR {
    VkStructureType          sType;
    const void*              pNext;
    uint32_t                 acquireCount;
    const VkDeviceMemory*    pAcquireSyncs;
    const uint64_t*          pAcquireKeys;
    const uint32_t*          pAcquireTimeouts;
    uint32_t                 releaseCount;
    const VkDeviceMemory*    pReleaseSyncs;
    const uint64_t*          pReleaseKeys;
} VkWin32KeyedMutexAcquireReleaseInfoKHR;

  • acquireCount is the number of entries in the pAcquireSyncs, pAcquireKeys, and pAcquireTimeoutMilliseconds arrays.

  • pAcquireSyncs is a pointer to an array of VkDeviceMemory objects which were imported from Direct3D 11 resources.

  • pAcquireKeys is a pointer to an array of mutex key values to wait for prior to beginning the submitted work. Entries refer to the keyed mutex associated with the corresponding entries in pAcquireSyncs.

  • pAcquireTimeoutMilliseconds is a pointer to an array of timeout values, in millisecond units, for each acquire specified in pAcquireKeys.

  • releaseCount is the number of entries in the pReleaseSyncs and pReleaseKeys arrays.

  • pReleaseSyncs is a pointer to an array of VkDeviceMemory objects which were imported from Direct3D 11 resources.

  • pReleaseKeys is a pointer to an array of mutex key values to set when the submitted work has completed. Entries refer to the keyed mutex associated with the corresponding entries in pReleaseSyncs.

Valid Usage

  • Each member of pAcquireSyncs and pReleaseSyncs must be a device memory object imported by setting VkImportMemoryWin32HandleInfoKHR::handleType to VK_EXTERNAL_MEMORY_HANDLE_TYPE_D3D11_TEXTURE_BIT or VK_EXTERNAL_MEMORY_HANDLE_TYPE_D3D11_TEXTURE_KMT_BIT

Valid Usage (Implicit)

  • sType must be VK_STRUCTURE_TYPE_WIN32_KEYED_MUTEX_ACQUIRE_RELEASE_INFO_KHR

  • If acquireCount is not 0, pAcquireSyncs must be a valid pointer to an array of acquireCount valid VkDeviceMemory handles

  • If acquireCount is not 0, pAcquireKeys must be a valid pointer to an array of acquireCount uint64_t values

  • If acquireCount is not 0, pAcquireTimeouts must be a valid pointer to an array of acquireCount uint32_t values

  • If releaseCount is not 0, pReleaseSyncs must be a valid pointer to an array of releaseCount valid VkDeviceMemory handles

  • If releaseCount is not 0, pReleaseKeys must be a valid pointer to an array of releaseCount uint64_t values

  • Both of the elements of pAcquireSyncs, and the elements of pReleaseSyncs that are valid handles of non-ignored parameters must have been created, allocated, or retrieved from the same VkDevice

When submitting work that operates on memory imported from a Direct3D 11 resource to a queue, the keyed mutex mechanism may be used in addition to Vulkan semaphores to synchronize the work. Keyed mutexes are a property of a properly created shareable Direct3D 11 resource. They can only be used if the imported resource was created with the D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX flag.

To acquire keyed mutexes before submitted work and/or release them after, add a VkWin32KeyedMutexAcquireReleaseInfoNV structure to the pNext chain of the VkSubmitInfo structure.

The VkWin32KeyedMutexAcquireReleaseInfoNV structure is defined as:

// Provided by VK_NV_win32_keyed_mutex
typedef struct VkWin32KeyedMutexAcquireReleaseInfoNV {
    VkStructureType          sType;
    const void*              pNext;
    uint32_t                 acquireCount;
    const VkDeviceMemory*    pAcquireSyncs;
    const uint64_t*          pAcquireKeys;
    const uint32_t*          pAcquireTimeoutMilliseconds;
    uint32_t                 releaseCount;
    const VkDeviceMemory*    pReleaseSyncs;
    const uint64_t*          pReleaseKeys;
} VkWin32KeyedMutexAcquireReleaseInfoNV;

  • acquireCount is the number of entries in the pAcquireSyncs, pAcquireKeys, and pAcquireTimeoutMilliseconds arrays.

  • pAcquireSyncs is a pointer to an array of VkDeviceMemory objects which were imported from Direct3D 11 resources.

  • pAcquireKeys is a pointer to an array of mutex key values to wait for prior to beginning the submitted work. Entries refer to the keyed mutex associated with the corresponding entries in pAcquireSyncs.

  • pAcquireTimeoutMilliseconds is a pointer to an array of timeout values, in millisecond units, for each acquire specified in pAcquireKeys.

  • releaseCount is the number of entries in the pReleaseSyncs and pReleaseKeys arrays.

  • pReleaseSyncs is a pointer to an array of VkDeviceMemory objects which were imported from Direct3D 11 resources.

  • pReleaseKeys is a pointer to an array of mutex key values to set when the submitted work has completed. Entries refer to the keyed mutex associated with the corresponding entries in pReleaseSyncs.

Valid Usage (Implicit)

  • sType must be VK_STRUCTURE_TYPE_WIN32_KEYED_MUTEX_ACQUIRE_RELEASE_INFO_NV

  • If acquireCount is not 0, pAcquireSyncs must be a valid pointer to an array of acquireCount valid VkDeviceMemory handles

  • If acquireCount is not 0, pAcquireKeys must be a valid pointer to an array of acquireCount uint64_t values

  • If acquireCount is not 0, pAcquireTimeoutMilliseconds must be a valid pointer to an array of acquireCount uint32_t values

  • If releaseCount is not 0, pReleaseSyncs must be a valid pointer to an array of releaseCount valid VkDeviceMemory handles

  • If releaseCount is not 0, pReleaseKeys must be a valid pointer to an array of releaseCount uint64_t values

  • Both of the elements of pAcquireSyncs, and the elements of pReleaseSyncs that are valid handles of non-ignored parameters must have been created, allocated, or retrieved from the same VkDevice

If the pNext chain of VkSubmitInfo includes a VkProtectedSubmitInfo structure, then the structure indicates whether the batch is protected. The VkProtectedSubmitInfo structure is defined as:

// Provided by VK_VERSION_1_1
typedef struct VkProtectedSubmitInfo {
    VkStructureType    sType;
    const void*        pNext;
    VkBool32           protectedSubmit;
} VkProtectedSubmitInfo;

  • protectedSubmit specifies whether the batch is protected. If protectedSubmit is VK_TRUE, the batch is protected. If protectedSubmit is VK_FALSE, the batch is unprotected. If the VkSubmitInfo::pNext chain does not include this structure, the batch is unprotected.

Valid Usage

  • If the protected memory feature is not enabled, protectedSubmit must not be VK_TRUE

  • If protectedSubmit is VK_TRUE, then each element of the pCommandBuffers array must be a protected command buffer

Valid Usage (Implicit)

  • sType must be VK_STRUCTURE_TYPE_PROTECTED_SUBMIT_INFO

If the pNext chain of VkSubmitInfo includes a VkDeviceGroupSubmitInfo structure, then that structure includes device indices and masks specifying which physical devices execute semaphore operations and command buffers.

The VkDeviceGroupSubmitInfo structure is defined as:

// Provided by VK_VERSION_1_1
typedef struct VkDeviceGroupSubmitInfo {
    VkStructureType    sType;
    const void*        pNext;
    uint32_t           waitSemaphoreCount;
    const uint32_t*    pWaitSemaphoreDeviceIndices;
    uint32_t           commandBufferCount;
    const uint32_t*    pCommandBufferDeviceMasks;
    uint32_t           signalSemaphoreCount;
    const uint32_t*    pSignalSemaphoreDeviceIndices;
} VkDeviceGroupSubmitInfo;

or the equivalent

// Provided by VK_KHR_device_group
typedef VkDeviceGroupSubmitInfo VkDeviceGroupSubmitInfoKHR;

  • sType is the type of this structure.

  • pNext is NULL or a pointer to a structure extending this structure.

  • waitSemaphoreCount is the number of elements in the pWaitSemaphoreDeviceIndices array.

  • pWaitSemaphoreDeviceIndices is a pointer to an array of waitSemaphoreCount device indices indicating which physical device executes the semaphore wait operation in the corresponding element of VkSubmitInfo::pWaitSemaphores.

  • commandBufferCount is the number of elements in the pCommandBufferDeviceMasks array.

  • pCommandBufferDeviceMasks is a pointer to an array of commandBufferCount device masks indicating which physical devices execute the command buffer in the corresponding element of VkSubmitInfo::pCommandBuffers. A physical device executes the command buffer if the corresponding bit is set in the mask.

  • signalSemaphoreCount is the number of elements in the pSignalSemaphoreDeviceIndices array.

  • pSignalSemaphoreDeviceIndices is a pointer to an array of signalSemaphoreCount device indices indicating which physical device executes the semaphore signal operation in the corresponding element of VkSubmitInfo::pSignalSemaphores.

If this structure is not present, semaphore operations and command buffers execute on device index zero.

Valid Usage

  • waitSemaphoreCount must equal VkSubmitInfo::waitSemaphoreCount

  • commandBufferCount must equal VkSubmitInfo::commandBufferCount

  • signalSemaphoreCount must equal VkSubmitInfo::signalSemaphoreCount

  • All elements of pWaitSemaphoreDeviceIndices and pSignalSemaphoreDeviceIndices must be valid device indices

  • All elements of pCommandBufferDeviceMasks must be valid device masks

Valid Usage (Implicit)

  • sType must be VK_STRUCTURE_TYPE_DEVICE_GROUP_SUBMIT_INFO

  • If waitSemaphoreCount is not 0, pWaitSemaphoreDeviceIndices must be a valid pointer to an array of waitSemaphoreCount uint32_t values

  • If commandBufferCount is not 0, pCommandBufferDeviceMasks must be a valid pointer to an array of commandBufferCount uint32_t values

  • If signalSemaphoreCount is not 0, pSignalSemaphoreDeviceIndices must be a valid pointer to an array of signalSemaphoreCount uint32_t values

If the pNext chain of VkSubmitInfo includes a VkPerformanceQuerySubmitInfoKHR structure, then the structure indicates which counter pass is active for the batch in that submit.

The VkPerformanceQuerySubmitInfoKHR structure is defined as:

// Provided by VK_KHR_performance_query
typedef struct VkPerformanceQuerySubmitInfoKHR {
    VkStructureType    sType;
    const void*        pNext;
    uint32_t           counterPassIndex;
} VkPerformanceQuerySubmitInfoKHR;

  • sType is the type of this structure.

  • pNext is NULL or a pointer to a structure extending this structure.

  • counterPassIndex specifies which counter pass index is active.

If the VkSubmitInfo::pNext chain does not include this structure, the batch defaults to use counter pass index 0.

Valid Usage
Valid Usage (Implicit)

  • sType must be VK_STRUCTURE_TYPE_PERFORMANCE_QUERY_SUBMIT_INFO_KHR

5.6. Queue Forward Progress

When using binary semaphores, the application must ensure that command buffer submissions will be able to complete without any subsequent operations by the application on any queue. After any call to vkQueueSubmit (or other queue operation), for every queued wait on a semaphore created with a VkSemaphoreType of VK_SEMAPHORE_TYPE_BINARY there must be a prior signal of that semaphore that will not be consumed by a different wait on the semaphore.

When using timeline semaphores, wait-before-signal behavior is well-defined and applications can submit work via vkQueueSubmit which defines a timeline semaphore wait operation before submitting a corresponding semaphore signal operation. For each timeline semaphore wait operation defined by a call to vkQueueSubmit, the application must ensure that a corresponding semaphore signal operation is executed before forward progress can be made.

Command buffers in the submission can include vkCmdWaitEvents commands that wait on events that will not be signaled by earlier commands in the queue. Such events must be signaled by the application using vkSetEvent, and the vkCmdWaitEvents commands that wait upon them must not be inside a render pass instance. The event must be set before the vkCmdWaitEvents command is executed.

Note

Implementations may have some tolerance for waiting on events to be set, but this is defined outside of the scope of Vulkan.

5.7. Secondary Command Buffer Execution

A secondary command buffer must not be directly submitted to a queue. Instead, secondary command buffers are recorded to execute as part of a primary command buffer with the command:

// Provided by VK_VERSION_1_0
void vkCmdExecuteCommands(
    VkCommandBuffer                             commandBuffer,
    uint32_t                                    commandBufferCount,
    const VkCommandBuffer*                      pCommandBuffers);

  • commandBuffer is a handle to a primary command buffer that the secondary command buffers are executed in.

  • commandBufferCount is the length of the pCommandBuffers array.

  • pCommandBuffers is a pointer to an array of commandBufferCount secondary command buffer handles, which are recorded to execute in the primary command buffer in the order they are listed in the array.

If any element of pCommandBuffers was not recorded with the VK_COMMAND_BUFFER_USAGE_SIMULTANEOUS_USE_BIT flag, and it was recorded into any other primary command buffer which is currently in the executable or recording state, that primary command buffer becomes invalid.

Valid Usage

  • Each element of pCommandBuffers must have been allocated with a level of VK_COMMAND_BUFFER_LEVEL_SECONDARY

  • Each element of pCommandBuffers must be in the pending or executable state

  • If any element of pCommandBuffers was not recorded with the VK_COMMAND_BUFFER_USAGE_SIMULTANEOUS_USE_BIT flag, it must not be in the pending state

  • If any element of pCommandBuffers was not recorded with the VK_COMMAND_BUFFER_USAGE_SIMULTANEOUS_USE_BIT flag, it must not have already been recorded to commandBuffer

  • If any element of pCommandBuffers was not recorded with the VK_COMMAND_BUFFER_USAGE_SIMULTANEOUS_USE_BIT flag, it must not appear more than once in pCommandBuffers

  • Each element of pCommandBuffers must have been allocated from a VkCommandPool that was created for the same queue family as the VkCommandPool from which commandBuffer was allocated

  • If vkCmdExecuteCommands is being called within a render pass instance, that render pass instance must have been begun with the contents parameter of vkCmdBeginRenderPass set to VK_SUBPASS_CONTENTS_SECONDARY_COMMAND_BUFFERS

  • If vkCmdExecuteCommands is being called within a render pass instance, each element of pCommandBuffers must have been recorded with the VK_COMMAND_BUFFER_USAGE_RENDER_PASS_CONTINUE_BIT

  • If vkCmdExecuteCommands is being called within a render pass instance, each element of pCommandBuffers must have been recorded with VkCommandBufferInheritanceInfo::subpass set to the index of the subpass which the given command buffer will be executed in

  • If vkCmdExecuteCommands is being called within a render pass instance, the render passes specified in the pBeginInfo->pInheritanceInfo->renderPass members of the vkBeginCommandBuffer commands used to begin recording each element of pCommandBuffers must be compatible with the current render pass

  • If vkCmdExecuteCommands is being called within a render pass instance, and any element of pCommandBuffers was recorded with VkCommandBufferInheritanceInfo::framebuffer not equal to VK_NULL_HANDLE, that VkFramebuffer must match the VkFramebuffer used in the current render pass instance

  • If vkCmdExecuteCommands is being called within a render pass instance that included VkRenderPassTransformBeginInfoQCOM in the pNext chain of VkRenderPassBeginInfo, then each element of pCommandBuffers must have been recorded with VkCommandBufferInheritanceRenderPassTransformInfoQCOM in the pNext chain of VkCommandBufferBeginInfo

  • If vkCmdExecuteCommands is being called within a render pass instance that included VkRenderPassTransformBeginInfoQCOM in the pNext chain of VkRenderPassBeginInfo, then each element of pCommandBuffers must have been recorded with VkCommandBufferInheritanceRenderPassTransformInfoQCOM::transform identical to VkRenderPassTransformBeginInfoQCOM::transform

  • If vkCmdExecuteCommands is being called within a render pass instance that included VkRenderPassTransformBeginInfoQCOM in the pNext chain of VkRenderPassBeginInfo, then each element of pCommandBuffers must have been recorded with VkCommandBufferInheritanceRenderPassTransformInfoQCOM::renderArea identical to VkRenderPassBeginInfo::renderArea

  • If vkCmdExecuteCommands is not being called within a render pass instance, each element of pCommandBuffers must not have been recorded with the VK_COMMAND_BUFFER_USAGE_RENDER_PASS_CONTINUE_BIT

  • If the inherited queries feature is not enabled, commandBuffer must not have any queries active

  • If commandBuffer has a VK_QUERY_TYPE_OCCLUSION query active, then each element of pCommandBuffers must have been recorded with VkCommandBufferInheritanceInfo::occlusionQueryEnable set to VK_TRUE

  • If commandBuffer has a VK_QUERY_TYPE_OCCLUSION query active, then each element of pCommandBuffers must have been recorded with VkCommandBufferInheritanceInfo::queryFlags having all bits set that are set for the query

  • If commandBuffer has a VK_QUERY_TYPE_PIPELINE_STATISTICS query active, then each element of pCommandBuffers must have been recorded with VkCommandBufferInheritanceInfo::pipelineStatistics having all bits set that are set in the VkQueryPool the query uses

  • Each element of pCommandBuffers must not begin any query types that are active in commandBuffer

  • If commandBuffer is a protected command buffer, then each element of pCommandBuffers must be a protected command buffer

  • If commandBuffer is an unprotected command buffer, then each element of pCommandBuffers must be an unprotected command buffer

  • This command must not be recorded when transform feedback is active

Valid Usage (Implicit)

  • commandBuffer must be a valid VkCommandBuffer handle

  • pCommandBuffers must be a valid pointer to an array of commandBufferCount valid VkCommandBuffer handles

  • commandBuffer must be in the recording state

  • The VkCommandPool that commandBuffer was allocated from must support transfer, graphics, or compute operations

  • commandBuffer must be a primary VkCommandBuffer

  • commandBufferCount must be greater than 0

  • Both of commandBuffer, and the elements of pCommandBuffers must have been created, allocated, or retrieved from the same VkDevice

Host Synchronization

  • Host access to commandBuffer must be externally synchronized

  • Host access to the VkCommandPool that commandBuffer was allocated from must be externally synchronized

Command Properties
Command Buffer Levels Render Pass Scope Supported Queue Types Pipeline Type

Primary

Both

Transfer
Graphics
Compute

5.8. Command Buffer Device Mask

Each command buffer has a piece of state storing the current device mask of the command buffer. This mask controls which physical devices within the logical device all subsequent commands will execute on, including state-setting commands, action commands, and synchronization commands.

Scissor, exclusive scissor, and viewport state (excluding the count of each) can be set to different values on each physical device (only when set as dynamic state), and each physical device will render using its local copy of the state. Other state is shared between physical devices, such that all physical devices use the most recently set values for the state. However, when recording an action command that uses a piece of state, the most recent command that set that state must have included all physical devices that execute the action command in its current device mask.

The command buffer’s device mask is orthogonal to the pCommandBufferDeviceMasks member of VkDeviceGroupSubmitInfo. Commands only execute on a physical device if the device index is set in both device masks.

If the pNext chain of VkCommandBufferBeginInfo includes a VkDeviceGroupCommandBufferBeginInfo structure, then that structure includes an initial device mask for the command buffer.

The VkDeviceGroupCommandBufferBeginInfo structure is defined as:

// Provided by VK_VERSION_1_1
typedef struct VkDeviceGroupCommandBufferBeginInfo {
    VkStructureType    sType;
    const void*        pNext;
    uint32_t           deviceMask;
} VkDeviceGroupCommandBufferBeginInfo;

or the equivalent

// Provided by VK_KHR_device_group
typedef VkDeviceGroupCommandBufferBeginInfo VkDeviceGroupCommandBufferBeginInfoKHR;

  • sType is the type of this structure.

  • pNext is NULL or a pointer to a structure extending this structure.

  • deviceMask is the initial value of the command buffer’s device mask.

The initial device mask also acts as an upper bound on the set of devices that can ever be in the device mask in the command buffer.

If this structure is not present, the initial value of a command buffer’s device mask is set to include all physical devices in the logical device when the command buffer begins recording.

Valid Usage

  • deviceMask must be a valid device mask value

  • deviceMask must not be zero

Valid Usage (Implicit)

  • sType must be VK_STRUCTURE_TYPE_DEVICE_GROUP_COMMAND_BUFFER_BEGIN_INFO

To update the current device mask of a command buffer, call:

// Provided by VK_VERSION_1_1
void vkCmdSetDeviceMask(
    VkCommandBuffer                             commandBuffer,
    uint32_t                                    deviceMask);

or the equivalent command

// Provided by VK_KHR_device_group
void vkCmdSetDeviceMaskKHR(
    VkCommandBuffer                             commandBuffer,
    uint32_t                                    deviceMask);

  • commandBuffer is command buffer whose current device mask is modified.

  • deviceMask is the new value of the current device mask.

deviceMask is used to filter out subsequent commands from executing on all physical devices whose bit indices are not set in the mask, except commands beginning a render pass instance, commands transitioning to the next subpass in the render pass instance, and commands ending a render pass instance, which always execute on the set of physical devices whose bit indices are included in the deviceMask member of the VkDeviceGroupRenderPassBeginInfo structure passed to the command beginning the corresponding render pass instance.

Valid Usage

  • deviceMask must be a valid device mask value

  • deviceMask must not be zero

  • deviceMask must not include any set bits that were not in the VkDeviceGroupCommandBufferBeginInfo::deviceMask value when the command buffer began recording

  • If vkCmdSetDeviceMask is called inside a render pass instance, deviceMask must not include any set bits that were not in the VkDeviceGroupRenderPassBeginInfo::deviceMask value when the render pass instance began recording

Valid Usage (Implicit)

  • commandBuffer must be a valid VkCommandBuffer handle

  • commandBuffer must be in the recording state

  • The VkCommandPool that commandBuffer was allocated from must support graphics, compute, or transfer operations

Host Synchronization

  • Host access to commandBuffer must be externally synchronized

  • Host access to the VkCommandPool that commandBuffer was allocated from must be externally synchronized

Command Properties
Command Buffer Levels Render Pass Scope Supported Queue Types Pipeline Type

Primary
Secondary

Both

Graphics
Compute
Transfer