33. Deferred Host Operations

Certain Vulkan commands are inherently expensive for the host CPU to execute. It is often desirable to offload such work onto background threads, and to parallelize the work across multiple CPUs. The concept of deferred operations allows applications and drivers to coordinate the execution of expensive host commands using an application-managed thread pool.

The VK_KHR_deferred_host_operations extension defines the infrastructure and usage patterns for deferrable commands, but does not specify any commands as deferrable. This is left to additional dependant extensions. Commands must not be deferred unless the deferral is specifically allowed by another extension which depends on VK_KHR_deferred_host_operations. This specification will refer to such extensions as deferral extensions.

33.1. Requesting Deferral

The VkDeferredOperationInfoKHR structure is defined as:

// Provided by VK_KHR_deferred_host_operations
typedef struct VkDeferredOperationInfoKHR {
    VkStructureType           sType;
    const void*               pNext;
    VkDeferredOperationKHR    operationHandle;
} VkDeferredOperationInfoKHR;

  • sType is the type of this structure.

  • pNext is NULL or a pointer to a structure extending this structure.

  • operationHandle is a handle to a tracking object to associate with the deferred operation.

The application can request deferral of an operation by adding this structure to the argument list of a command or by providing this in the pNext chain of a relevant structure for an operation when the corresponding command is invoked. If this structure is not present, no deferral is requested. If operationHandle is VK_NULL_HANDLE, no deferral is requested and the command proceeds as if no VkDeferredOperationInfoKHR structure was provided.

When an application requests an operation deferral, the implementation may defer the operation. When deferral is requested and the implementation defers any operation, the implementation must return VK_OPERATION_DEFERRED_KHR as the success code if no errors occurred. When deferral is requested, the implementation should defer the operation when the workload is significant, however if the implementation chooses not to defer any of the requested operations and instead executes all of them immediately, the implementation must return VK_OPERATION_NOT_DEFERRED_KHR as the success code if no errors occurred.

A deferred operation is created complete with an initial result value of VK_SUCCESS. The deferred operation becomes pending when an operation has been successfully deferred with that operationHandle.

A deferred operation is considered pending until the deferred operation completes. A pending deferred operation becomes complete when it has been fully executed by one or more threads. Pending deferred operations will never complete until they are joined by an application thread, using vkDeferredOperationJoinKHR. Applications can join multiple threads to the same deferred operation, enabling concurrent execution of subtasks within that operation.

The application can query the status of a VkDeferredOperationKHR using the vkGetDeferredOperationMaxConcurrencyKHR or vkGetDeferredOperationResultKHR commands.

From the perspective of other commands - parameters to the original command that are externally synchronized must not be accessed before the deferred operation completes, and the result of the deferred operation (e.g. object creation) are not considered complete until the deferred operation completes.

If the deferred operation is one which creates an object (for example, a pipeline object), the implementation must allocate that object as it normally would, and return a valid handle to the application. This object is a pending object, and must not be used by the application until the deferred operation is completed (unless otherwise specified by the deferral extension). When the deferred operation is complete, the application should call vkGetDeferredOperationResultKHR to obtain the result of the operation. If vkGetDeferredOperationResultKHR indicates failure, the application must destroy the pending object using an appropriate command, so that the implementation has an opportunity to recover the handle. The application must not perform this destruction until the deferred operation is complete. Construction of the pending object uses the same allocator which would have been used if the operation had not been deferred.

Valid Usage

  • Any previous deferred operation that was associated with operationHandle must be complete

Valid Usage (Implicit)

  • sType must be VK_STRUCTURE_TYPE_DEFERRED_OPERATION_INFO_KHR

33.2. Deferred Host Operations API

The VkDeferredOperationKHR handle is defined as:

// Provided by VK_KHR_deferred_host_operations
VK_DEFINE_NON_DISPATCHABLE_HANDLE(VkDeferredOperationKHR)

This handle refers to a tracking structure which manages the execution state for a deferred command.

To construct the tracking object for a deferred command, call:

// Provided by VK_KHR_deferred_host_operations
VkResult vkCreateDeferredOperationKHR(
    VkDevice                                    device,
    const VkAllocationCallbacks*                pAllocator,
    VkDeferredOperationKHR*                     pDeferredOperation);

  • device is the device which owns operation.

  • pAllocator controls host memory allocation as described in the Memory Allocation chapter.

  • pDeferredOperation is a pointer to a handle in which the created VkDeferredOperationKHR is returned.

Valid Usage (Implicit)
Return Codes
Success

  • VK_SUCCESS

Failure

  • VK_ERROR_OUT_OF_HOST_MEMORY

To assign a thread to a deferred operation, call:

// Provided by VK_KHR_deferred_host_operations
VkResult vkDeferredOperationJoinKHR(
    VkDevice                                    device,
    VkDeferredOperationKHR                      operation);

  • device is the device which owns operation.

  • operation is the deferred operation that the calling thread should work on.

The vkDeferredOperationJoinKHR command will execute a portion of the deferred operation on the calling thread.

The return value will be one of the following:

  • A return value of VK_SUCCESS indicates that operation is complete. The application should use vkGetDeferredOperationResultKHR to retrieve the result of operation.

  • A return value of VK_THREAD_DONE_KHR indicates that the deferred operation is not complete, but there is no work remaining to assign to threads. Future calls to vkDeferredOperationJoinKHR are not necessary and will simply harm performance. This situation may occur when other threads executing vkDeferredOperationJoinKHR are about to complete operation, and the implementation is unable to partition the workload any further.

  • A return value of VK_THREAD_IDLE_KHR indicates that the deferred operation is not complete, and there is no work for the thread to do at the time of the call. This situation may occur if the operation encounters a temporary reduction in parallelism. By returning VK_THREAD_IDLE_KHR, the implementation is signaling that it expects that more opportunities for parallelism will emerge as execution progresses, and that future calls to vkDeferredOperationJoinKHR can be beneficial. In the meantime, the application can perform other work on the calling thread.

Implementations must guarantee forward progress by enforcing the following invariants:

  1. If only one thread has invoked vkDeferredOperationJoinKHR on a given operation, that thread must execute the operation to completion and return VK_SUCCESS.

  2. If multiple threads have concurrently invoked vkDeferredOperationJoinKHR on the same operation, then at least one of them must complete the operation and return VK_SUCCESS.

Valid Usage (Implicit)

  • device must be a valid VkDevice handle

  • operation must be a valid VkDeferredOperationKHR handle

  • operation must have been created, allocated, or retrieved from device

Return Codes
Success

  • VK_SUCCESS

  • VK_THREAD_DONE_KHR

  • VK_THREAD_IDLE_KHR

Failure

  • VK_ERROR_OUT_OF_HOST_MEMORY

  • VK_ERROR_OUT_OF_DEVICE_MEMORY

When a deferred operation is completed, the application can destroy the tracking object by calling:

// Provided by VK_KHR_deferred_host_operations
void vkDestroyDeferredOperationKHR(
    VkDevice                                    device,
    VkDeferredOperationKHR                      operation,
    const VkAllocationCallbacks*                pAllocator);

  • device is the device which owns operation.

  • operation is the completed operation to be destroyed.

  • pAllocator controls host memory allocation as described in the Memory Allocation chapter.

Valid Usage

  • If VkAllocationCallbacks were provided when operation was created, a compatible set of callbacks must be provided here

  • If no VkAllocationCallbacks were provided when operation was created, pAllocator must be NULL

  • operation must be completed

Valid Usage (Implicit)

  • device must be a valid VkDevice handle

  • If operation is not VK_NULL_HANDLE, operation must be a valid VkDeferredOperationKHR handle

  • If pAllocator is not NULL, pAllocator must be a valid pointer to a valid VkAllocationCallbacks structure

  • If operation is a valid handle, it must have been created, allocated, or retrieved from device

Host Synchronization

  • Host access to operation must be externally synchronized

To query the number of additional threads that can usefully be joined to a deferred operation, call:

// Provided by VK_KHR_deferred_host_operations
uint32_t vkGetDeferredOperationMaxConcurrencyKHR(
    VkDevice                                    device,
    VkDeferredOperationKHR                      operation);

  • device is the device which owns operation.

  • operation is the deferred operation to be queried.

The returned value is the maximum number of threads that can usefully execute a deferred operation concurrently, reported for the state of the deferred operation at the point this command is called. This value is intended to be used to better schedule work onto available threads. Applications can join any number of threads to the deferred operation and expect it to eventually complete, though excessive joins may return VK_THREAD_DONE_KHR immediately, performing no useful work.

If operation is complete, vkGetDeferredOperationMaxConcurrencyKHR returns zero.

If operation is currently joined to any threads, the value returned by this command may immediately be out of date.

If operation is pending, implementations must not return zero unless at least one thread is currently executing vkDeferredOperationJoinKHR on operation. If there are such threads, the implementation should return an estimate of the number of additional threads which it could profitably use.

Implementations may return 232-1 to indicate that the maximum concurrency is unknown and cannot be easily derived. Implementations may return values larger than the maximum concurrency available on the host CPU. In these situations, an application should clamp the return value rather than oversubscribing the machine.

Note

The recommended usage pattern for applications is to query this value once, after deferral, and schedule no more than the specified number of threads to join the operation. Each time a joined thread receives VK_THREAD_IDLE_KHR, the application should schedule an additional join at some point in the future, but is not required to do so.
Valid Usage (Implicit)

  • device must be a valid VkDevice handle

  • operation must be a valid VkDeferredOperationKHR handle

  • operation must have been created, allocated, or retrieved from device

The vkGetDeferredOperationResultKHR function is defined as:

// Provided by VK_KHR_deferred_host_operations
VkResult vkGetDeferredOperationResultKHR(
    VkDevice                                    device,
    VkDeferredOperationKHR                      operation);

  • device is the device which owns operation.

  • operation is the operation whose deferred result is being queried.

If the deferred operation is pending, vkGetDeferredOperationResultKHR returns VK_NOT_READY.

If no command has been deferred on operation, vkGetDeferredOperationResultKHR returns VK_SUCCESS.

Otherwise, it returns the result of the previous deferred operation. This value must be one of the VkResult values which could have been returned by the original command if the operation had not been deferred.

Valid Usage (Implicit)

  • device must be a valid VkDevice handle

  • operation must be a valid VkDeferredOperationKHR handle

  • operation must have been created, allocated, or retrieved from device

Return Codes
Success

  • VK_SUCCESS

  • VK_NOT_READY