Vulkan pipeline barriers visualized Published: 6/22/2026
GPU side synchronization and pipeline barriers are among the least understood topics when it comes to Vulkan development, especially among developers who are new to the API. This blog post aims to provide visual examples for how they work and how you can correctly configure their parameters yourself.
Worth noting, instead of the superseded default synchronization functions, I’ll
be using the
synchronization2
feature (from now on referred to as sync2 for brevity), which should be
basically ubiquitous when targeting Vulkan 1.3 or newer, with the extension form
(VK_KHR_synchronization2) filling in for 1.2 capable devices with a similar
support rate. The main difference between the legacy API and sync2 are:
- The source and destination pipeline stages were moved into the barrier structs, meaning it is no longer necessary to sacrifice the precision of synchronization scopes when trying to synchronize all resources across all pipelines with just a single command.
- The
TOP_OF_PIPEandBOTTOM_OF_PIPEpipeline stages were deprecated and fully replaced byNONEandALL_COMMANDS(how they map to each other depends on the context). - Both pipeline stages and access types were extended to 64 bit flags, giving the developer much finer control over synchronization (in theory).
In the guide whenever I feel it’s clear from the context, I’ll shorten the names
of the constants by omitting the name of the enum and the “BIT” suffix, e.g.
instead of VK_PIPELINE_STAGE_2_FRAGMENT_SHADER_BIT I’ll write
FRAGMENT_SHADER. In my opinion this makes the text way more readable. I also
won’t be including stages that aren’t core yet.
Alternative guides
If at any point you feel that my explanations aren’t good enough or you have moral objections against sync2, feel free to bail and use one of the following resources:
- Yet another blog explaining Vulkan synchronization
- Synchronization and Cache Control (Official Vulkan docs)
I personally recommend reading them both in that order.
Alternatively feel free to mention any issues you find in the issues page for the repository of my website.
Synchronization components
Vulkan synchronization is made up of the following 4 parts:
Pipeline stages
Pipeline stages represent a specific part of command execution, for example:
COLOR_ATTACHMENT_OUTPUT: The part of the pipeline when the GPU takes the outputs from the fragment shader and applies them onto the attachment images.COPY: The stage of the pipeline where copy commands are executed, these are responsible for GPU->GPU copies.FRAGMENT_SHADER: The part of the pipeline when the fragment shader is executed.
Some values in the VkPipelineStageFlagBits2 enum are helpers representing a
collection of stages. For instance instead of COPY from the example above,
ALL_TRANSFER could be specified, which alongside the COPY pipeline stage
would also include the stages responsible for BLIT, RESOLVE and CLEAR
operations.
Pipeline stage flags also contain a special NONE value which references none
of the pipeline stages and is also the default when zero initializing.
More about
TOP_OF_PIPEandBOTTOM_OF_PIPE:Before sync2 these two were the main way to to mark that you want to target the very start or very end of the pipeline respectively. With sync2 they were deprecated, most likely due to it being a bit confusing which means which end.
Their replacements are
ALL_COMMANDSandNONEdepending on the usage, the following table contains the relationship based on the combination of which stage was used and whether in the source or destination stage mask
srcStageMask dstStageMask TOP_OF_PIPE NONE ALL_COMMANDS BOTTOM_OF_PIPE ALL_COMMANDS NONE The difference comes from the fact that
TOP_OF_PIPEandBOTTOM_OF_PIPErepresented “virtual” stages right before the first and right after the last actual pipeline stages in the current pipeline, whileALL_COMMANDSandNONErepresent a union of all or none of the stages.
Access types
Access types denote the various read and write memory operations (or in other words, accesses) you can do to a resource. Some examples are:
COLOR_ATTACHMENT_WRITE: This represents the GPU taking the outputs from the fragment shader and applies them onto the attachment images, possibly doing blending operations.HOST_WRITE: This represents the host (usually the CPU) writing to a location in the GPU memory.SHADER_STORAGE_WRITE: This represents a shader program writing to a storage resource (e.g. a storage image or a storage buffer).
Similarly to pipeline stages, access flags also include the special NONE valu,
which tells the driver that you don’t expect or care about any actual accesses
to the resource. Since this has the bit pattern 0, this will be the default
value when zero initializing structs.
Since access types represent actual operations done to a resource, naturally
certain access types might only be performed by specific pipeline stages, for
instance an access of type COLOR_ATTACHMENT_WRITE can only happen during the
COLOR_ATTACHMENT_OUTPUT pipeline stage and the COMPUTE_SHADER pipeline stage
will never access a resource with the VERTEX_ATTRIBUTE_READ access type. The
full table for what stages support specific access types is confusingly only
available under the
documentation for the legacy VkAccessFlagBits
enum, despite also containing sync2 values.
Pipelines
Importantly when I mention “pipeline” in this guide, I’m not talking about VkPipeline, but instead the list of steps taken by the GPU to execute a specific command. For instance the rasterization pipeline is the list of steps it takes to go from raw index and vertex data to pixels on a texture. In some sense this is closer to what people refer to with the “fixed function” part of gpu processes.
Pipelines are the way the specification defines an ordering between the various pipeline stages. There are over a dozen different ones (some of which are vendor specific), but the 3 most important are the graphics primitive pipeline (which is different from the mesh shader based graphics mesh pipeline) with the following pipeline stages:
DRAW_INDIRECTINDEX_INPUTVERTEX_ATTRIBUTE_INPUTVERTEX_SHADERTESSELLATION_CONTROL_SHADERTESSELLATION_EVALUATION_SHADERGEOMETRY_SHADEREARLY_FRAGMENT_TESTSFRAGMENT_SHADERLATE_FRAGMENT_TESTSCOLOR_ATTACHMENT_OUTPUT
the compute pipeline with the following stages:
DRAW_INDIRECT(this is not an error, this stage includes all indirect parameter fetches, it’s just badly named)COMPUTE_SHADER
and the transfer pipeline, which contains the COPY, BLIT, RESOLVE and
CLEAR stages, but the ordering between these isn’t defined since no operation
exists that can do work in 2 or more of these during execution.
The ordering between the stages in a pipeline is weak, the specification only guarantees that stages can only start if all previous stages have started already and can only finish if all previous stages have done so as well, but doesn’t guarantee that only 1 stage will be executing operations at any point.
Commands
The only thing the specification guarantees about command execution is that a command can only start once all commands before it have started as well. This is also very weak, as even launching all commands at the exact same time can satisfy this requirement.
It’s also worth noting, that this is purely semantics to make synchronization easier to define, in reality the GPU might reorder execution as it see fits as long as it doesn’t break any other synchronization guarantees (in other words, if it can prove that there exists an alternative ordering where the commands start in the specification defined order and produce the same result).
Notably there’s also no guaranteed implicit synchronization between command buffers. Think of them more like packages that you send off to the driver, which unpacks them and places the contents into an internal command queue. This makes splitting command lists into multiple buffers only useful if it’s done as a way to enable multi-threaded command recording or for the sake of easier development.
An exception to the rules are the drawing commands happening within the same render pass, primitives drawn by these must pass through the graphics pipeline in order (so if you draw primitive A then primitive B and both pass the various fragment tests, then A can’t ever appear “on top” of B).
Hazards
In multithreading contexts, hazards are when the state or contents of a resource becomes ambigious due to it being accessed from multiple threads in an incompatible way. For our purposes, this can manifest in 3 different ways:
Write-After-Read (WAR)
Write-After-read happens when a command is reading from an image that a subsequent command will write to.
Example:
- Image starts out red.
- First command dispatches a compute shader that copies the contents of the image to the screen.
- Second command dispatches a compute shader that fills the image with a blue color.
In these simulations the black square with a character in it represents operations done by the first command and the similar white square represents operations done by the second one. When the square contains an “R”, it means that the color under it is read into an internal storage register and “W” means some color (potentially the one contained in the register) is written to the pixel.
Read-After-Write (RAW)
This happens when one command writes to a resource and a subsequent command reads from it.
Example:
- Image starts out red.
- First command dispatches a compute shader that fills the image with blue.
- Second command dispatches a compute shader that copies the contents of the image to the screen (for demonstration purposes the copying happens backwards).
Write-After-Write (WAW)
This happens when two subsequent commands are writing to the same resource.
Example:
- First command dispatches a compute shader that fills the screen in red.
- Second command dispatches a compute shader that fills the screen in blue (once again for demonstration purposes this happens backwards).
Synchronization with pipeline barriers
A pipeline barrier is a Vulkan synchronization primitive (specifically one that does GPU-GPU synchronization within a single queue as opposed to something like a fence which handles CPU-GPU sync). With sync2 the function header for them is
void vkCmdPipelineBarrier2(
VkCommandBuffer commandBuffer,
const VkDependencyInfo *dependencyInfo
);
Most of the information is in VkDependencyInfo, that looks like this:
typedef struct VkDependencyInfo {
VkStructureType sType;
const void *pNext;
VkDependencyFlags dependencyFlags;
uint32_t memoryBarrierCount;
const VkMemoryBarrier2 *pMemoryBarriers;
uint32_t bufferMemoryBarrierCount;
const VkBufferMemoryBarrier2 *pBufferMemoryBarriers;
uint32_t imageMemoryBarrierCount;
const VkImageMemoryBarrier2 *pImageMemoryBarriers;
} VkDependencyInfo;
These are in turn mainly made up of 3 types of barriers:
- Memory barriers, that synchronize the state of the full GPU memory.
- Buffer memory barriers, that work similarly to memory barriers, but limit the scope of synchronization to a specific buffer or even a specific region of a buffer.
- Image memory barriers that (similarly to buffer memory barriers) can restrict the scope of synchronization to the memory of a specific image or part of an image, but also have the very important job of handling layout transitions.
VkBufferMemoryBarrier2 and VkImageMemoryBarrier2 are basically supersets of
VkMemoryBarrier2 with extra parameters, so I’ll be discussing them later.
Memory barriers
This struct has the following fields:
typedef struct VkMemoryBarrier2 {
VkStructureType sType;
const void *pNext;
VkPipelineStageFlags2 srcStageMask;
VkAccessFlags2 srcAccessMask;
VkPipelineStageFlags2 dstStageMask;
VkAccessFlags2 dstAccessMask;
} VkMemoryBarrier2;
I’ll assume you already know how to set up sType and pNext. The rest are
what really matter here.
As the variable names and my strategic use of empty space might suggest, the remaining 4 parameters can be divided up into 2 groups, the first two define where the writing operations will be coming from and the second two tell the driver where these changes should propagate to.
Let’s disregard the access masks for the next subchapter to first understand what the stage masks represent.
Synchronization scopes and execution barriers
Barriers that only define the stage masks (and leave the access masks as NONE)
are so-called execution barriers.
The two stage masks define two so-called synchronization scopes, the specification refers to these as the “first” and “second” scopes, though I personally prefer “source” and “destination”. These are the set of pipeline stages the synchronization will happen between.
A pipeline barrier splits execution into three sections:
- The barrier cannot execute until all commands submitted before it have exited the first/source synchronization scope. The commands after the barrier can start execution, however they are not allowed to enter the second/destination synchronization scope.
- At this point the barrier executes (without access masks this happens basically instantly as there’s nothing else to do, otherwise read and write operations might have to take place here).
- Once the pipeline barrier finishes, all execution limits created by it are lifted and the commands that were submitted after it can finally enter the second scope.
An important note about the two synchronization scopes: Since
- a command can only start and finish pipeline stages in order,
- commands can only start after the previous commands in the queue have started as well
- and future commands can only enter stages in
dstStageMaskwhen the previous commands have exited all stages insrcStageMask
if you include a stage in srcStageMask, the synchronization scope will consist
of that stage and any stages that are before it. Similarly the scope defined by
dstStageMask will include the stages specified in the mask and all stages
after the in their respective pipelines. This means two tings:
- If both stage masks contain the same pipeline stage, they will overlap.
- For execution barriers it doesn’t make sense to include more than one stage in the stage masks from the same pipeline.
Execution barreirs are actually already enough to get around any WAR (write-after-read) hazards, since in those cases its enough to hold up the execution of the second command until after the first one had time to finish.
For a concrete example: Lets say you have an image and the last operation on it
was a read from a fragment shader and now you want to completely override it in
a draw command (most gbuffers go through this exact scenario regularly for
instance). You also happen to be on hardware where using the GENERAL image
layout is optimal for both of these situations (this will be explained later).
This is very clearly a WAR hazard, the second operation might start before the
first one finished reading from the texture, causing the output to become
garbage. To stop this from happening, we need an execution barrier between the
two commands:
// First operation that reads from the image
vkCmdDraw(commandBuffer, ...);
{
VkMemoryBarrier2 barrier {
.sType = VK_STRUCTURE_TYPE_MEMORY_BARRIER_2,
// We're reading from the image in the fragment shader
.srcStageMask = VK_PIPELINE_STAGE_2_FRAGMENT_SHADER_BIT,
// We'll be writing to it as a color attachment
.dstStageMask = VK_PIPELINE_STAGE_2_COLOR_ATTACHMENT_OUTPUT_BIT,
};
VkDependencyInfo dependencyInfo {
.sType = VK_STRUCTURE_TYPE_DEPENDENCY_INFO,
.memoryBarrierCount = 1,
.pMemoryBarriers = &barrier;
};
vkCmdPipelineBarrier2(commandBuffer, &dependencyInfo);
}
// Second operation that writes to the image
vkCmdDraw(commandBuffer, ...);
Here’s a visualization of this setup:
Reading this diagram:
The pipelines are laid out horizontally. On the bottom half of the graphic the various ribbons represent commands (drawn using a blueish tone) or barriers (the green ones), their submission order is from top to bottom.
The horizontal space is divided up into sections using the dotted lines, each of these represents a pipeline stage, the name of which is placed in the boxes above the commands. To aid readability on mobile devices, I only include some of the stages from their respective pipelines.
As the command executions are simulated, their ribbons fill up from left to right, this denotes them finishing their tasks in the different pipeline stages. To illustrate commands being able to finish in any order, I added some randomness to their execution speeds.
Barriers show their source and destination stages, the former with a “key” icon and the latter using a “lock”. All commands after the barriers must stop before the locks until commands before them passed all the key checkpoints.
Notice how in this example the second command can start execution immediately,
but it has to stop before it reaches the COLOR_ATTACHMENT_OUTPUT stage until
the first command finishes the FRAGMENT_SHADER stage.
The way I like to imagine this is as a lock and key system, where the keys are
the commands submitted before the barrier leaving the stages enumerated by
srcStageMask and the locks are the halting points defined by dstStageMask.
This is how I marked this relationship on the diagram as well.
Access masks, availability and visibility
An execution dependency is not enough to synchronize all memory operations, there are multiple reasons for this:
- With caches we minmaxed localized read and write speeds so hard, one could argue it’s incorrect to call such a storage medium “random-access-memory”. These caches need time to be flushed to main memory, which might not happen immediately after the operation that wrote to them finishes.
- Some GPUs (mainly older ones when talking about desktop hardware) have to decompress data into global memory before it becomes usable.
The only time these can be waited on is while executing the pipeline barrier and access masks are used to tell the driver what kind of operations we wish to include in this.
With access masks we introduce a new scope type: the access scope. Unlike synchronization scopes, these are exact, the barrier will only care about the accesses happening in the stages explicitly defined in the stage masks.
Example: if the source stage mask is
GEOMETRY_SHADER | FRAGMENT_SHADERand the source access mask isSHADER_READ | SHADER_WRITE, the first access scope will only include the following 4:
- A read happening in the geometry shader
- A write happening in the geometry shader
- A read happening in the fragment shader
- A write happening in the fragment shader
So even though the vertex shader is included in the source synchronization scope (since it’s earlier than the geometry shader), neither a read, nor a write operation will be taken into consideration from within it.
Before the results of a write operation can be accessed by other read operations, those writes have to be first made available, then visible to them. Availability basically means that a write has finished and the results were stored to memory somewhere, but other operations might not be able to read from them yet. What writes should become available after the pipeline barrier is defined by the source access scope. Visibility on the other hand ensures that future reads and writes have access to the new data, what specific operations are included in this is specified by the destination access mask.
With access masks the steps taken by a pipeline barrier are extended, the new steps are marked in bold:
- The barrier cannot execute until all commands submitted before it have exited the first/source synchronization scope. The commands after the barrier can start execution, however they are not allowed to enter the second/destination synchronization scope.
- At this point the barrier executes.
- All memory writes done in the first access scope are made available.
- All memory writes made available by the previous step are made visible to the second access scope.
- Once the pipeline barrier command finishes, all execution limits created by it are lifted and the commands that were submitted after it can continue past their stopping points.
In simple(-ish) terms, the source access masks tell the driver what memory
operations will be writing to the resource and the destination access types tell
it what operations will need to access the resource after these changes. This
means that it never makes sense to include _READ type accesses in
srcAccessMask since these don’t cause any changes.
A common pattern when it comes to synchronization is having two compute shaders, one of which needs to work on the data the former generated. This is a RAW hazard normally, so we need to insert a barrier.
// First dispatch writes to memory
vkCmdDispatch(commandBuffer, ...);
{
VkMemoryBarrier2 barrier {
.sType = VK_STRUCTURE_TYPE_MEMORY_BARRIER_2,
.srcStageMask = VK_PIPELINE_STAGE_2_COMPUTE_SHADER_BIT,
// SHADER_WRITE includes all write accesses done from within shaders
.srcAccessMask = VK_ACCESS_2_SHADER_WRITE_BIT,
.dstStageMask = VK_PIPELINE_STAGE_2_COMPUTE_SHADER_BIT,
// Similarly SHADER_READ includes all read accesses done from within shaders
.dstAccessMask = VK_ACCESS_2_SHADER_READ_BIT,
};
VkDependencyInfo dependencyInfo {
.sType = VK_STRUCTURE_TYPE_DEPENDENCY_INFO,
.memoryBarrierCount = 1,
.pMemoryBarriers = &barrier;
};
vkCmdPipelineBarrier2(commandBuffer, &dependencyInfo);
}
// Second dispatch reads from it
vkCmdDispatch(commandBuffer, ...);
Buffer memory barriers
The buffer memory barrier is a superset of the regular memory barrier:
typedef struct VkBufferMemoryBarrier2 {
VkStructureType sType;
const void* pNext;
VkPipelineStageFlags2 srcStageMask;
VkAccessFlags2 srcAccessMask;
VkPipelineStageFlags2 dstStageMask;
VkAccessFlags2 dstAccessMask;
uint32_t srcQueueFamilyIndex;
uint32_t dstQueueFamilyIndex;
VkBuffer buffer;
VkDeviceSize offset;
VkDeviceSize size;
} VkBufferMemoryBarrier2;
The first few fields serve the exact same purposes as they do in
VkMemoryBarrier2. The last three are also pretty self-explanatory, these are
what let you specify what memory region to synchronize for, down to a specific
part of the buffer.
The remaining two fields (srcQueueFamilyIndex and dstQueueFamilyIndex) are
for ownership transfer, which has its own section in this guide later.
Disregarding ownership transfers, buffer memory barriers provide the same utility as regular memory barriers, with the added complexity of potentially letting you limit what memory section the driver will have to be concerned with.
Image memory barriers
typedef struct VkImageMemoryBarrier2 {
VkStructureType sType;
const void* pNext;
VkPipelineStageFlags2 srcStageMask;
VkAccessFlags2 srcAccessMask;
VkPipelineStageFlags2 dstStageMask;
VkAccessFlags2 dstAccessMask;
VkImageLayout oldLayout;
VkImageLayout newLayout;
uint32_t srcQueueFamilyIndex;
uint32_t dstQueueFamilyIndex;
VkImage image;
VkImageSubresourceRange subresourceRange;
} VkImageMemoryBarrier2;
I won’t be reiterating myself when it comes to repeated fields
(subresourceRange is similar enough to offset and size in
VkBufferMemoryBarrier2), but we do have two new ones: oldLayout and
newLayout. These are what facilitate layout transitions.
Image layouts are meant to give hints to drivers for what an image resource will be used for. This can be important to allow for certain memory and bandwidth optimizations, for instance for certain usages the GPU might be able to compress the image.
Layout transitions work the same way as any other barriers, except you also
define the layout of the image from before the barrier (oldLayout) and the
layout you want it to be after (newLayout). Most layouts are designated for
specific use cases (e.g. COLOR_ATTACHMENT_OPTIMAL is for images that you’ll be
using as a color attachment, TRANSFER_SRC_OPTIMAL and TRANSFER_DST_OPTIMAL
are for reading from or writing to respectively during a transfer operation and
PRESENT_SRC_KHR is for presentation). There are also two special flags:
UNDEFINEDis only usable ininitialLayoutwhen creating a new image and asoldLayoutin barriers. The former lets you opt out of any kind of initialization during image creation and using it for the latter tells the driver, that you don’t care about the contents of the image anymore and it can do with it as it wishes. You could imagine that if the image needed to be decompressed to make the data properly readable again, it would be quite a large performance gain to skip that step if we don’t care about the contents anymore because we’ll be overwriting them anyways.GENERALas the name might imply is applicable to most usages with some exceptions (mainly around external systems, like presentation).
With modern features such as dynamic rendering, image layouts don’t really
change on their own, so if you don’t have a rendergraph-like system that keeps
track of them implicitly, it’s usually enough to cache the current layout of an
image and only update it to newLayout whenever you add a pipeline barrier.
class SmartImage {
public:
SmartImage() {
// ...
}
void transition(VkCommandBuffer commandBuffer, VkImageLayout to) {
VkImageMemoryBarrier2 barrier {
// ...
.oldLayout = this->currentLayout;
.newLayout = to;
};
VkDependencyInfo dependencyInfo {
// ...
.bufferMemoryBarrierCount = 1,
.pBufferMemoryBarriers = &barrier,
};
vkCmdPipelineBarrier(commandBuffer, &dependencyInfo);
this->currentLayout = to;
}
private:
VkImage handle;
VkImageLayout currentLayout = VK_IMAGE_LAYOUT_UNDEFINED;
}
Of course this can break down in multi-threaded environments or if you reuse command buffers.
Resource ownership
Most simple Vulkan applications should be able to get away with using a single queue for everything, but at some point it might be necessary to branch out to different ones, for example to make use of dedicated transfer or compute queues for their async capabilities.
When creating resources with the EXCLUSIVE sharing mode, we tell the driver
that these will be firmly attached to a single queue each (the initial owner
being determined by whichever queue uses it first) and we’ll be managing all
ownership transfers manually.
A complete ownership transfer is made up of two parts:
- The queue that owns the resource (in other words, the source queue) must release its ownership over the resource.
- The target queue (destination) must then acquire the resource.
Each of these is done with a pipeline barrier where srcQueueFamilyIndex is the
index of the family the source queue belongs to and dstQueueFamilyIndex is the
index of the family the destination queue is part of. As pipeline barriers don’t
synchronize between queues, it’s important to make sure these come in this exact
order using semaphores.
// First release the resource from the first queue
{
VkBufferMemoryBarrier2 barrier {
// ...
.srcQueueFamilyIndex = firstQueueFamilyIndex,
.srcQueueFamilyIndex = secondQueueFamilyIndex,
};
VkDependencyInfo dependencyInfo {
// ...
.bufferMemoryBarrierCount = 1,
.pBufferMemoryBarriers = &barrier,
};
vkCmdPipelineBarrier(firstCommandBuffer, &dependencyInfo);
VkSubmitInfo submitInfo {
// ...
.commandBufferCount = 1,
.pCommandBuffer = firstCommandBuffer,
.signalSemaphoreCount = 1,
.pSignalSemaphores = &ownershipTransferSemaphore,
};
vkQueueSubmit(firstQueue, 1, &submitInfo, VK_NULL_HANDLE);
}
// Next acquire it for the second queue
{
VkBufferMemoryBarrier2 barrier {
// ...
.srcQueueFamilyIndex = firstQueueFamilyIndex,
.srcQueueFamilyIndex = secondQueueFamilyIndex,
};
VkDependencyInfo dependencyInfo {
// ...
.bufferMemoryBarrierCount = 1,
.pBufferMemoryBarriers = &barrier,
};
vkCmdPipelineBarrier(secondCommandBuffer, &dependencyInfo);
VkSubmitInfo submitInfo {
// ...
.commandBufferCount = 1,
.pCommandBuffer = secondCommandBuffer,
.waitSemaphoreCount = 1,
.pWaitSemaphores = &ownershipTransferSemaphore,
};
vkQueueSubmit(secondQueue, 1, &submitInfo, VK_NULL_HANDLE);
}
Releasing and acquiring resources both can be done on their own without the other in some special cases.
Releasing a resource then never explicitly reacquiring it puts it into a condition where no queue has ownership over it and its contents are in an undefined state, which is close to what you got when you initially created it. If any queue uses the resource at this point, they will gain ownership over it, but the contents will be lost. This might be useful if for example one queue is responsible for completely overriding a resource every frame, then another queue reads from it and needs to hand it back to the original one at the end.
Acquiring without a prior release is an even more niche operation, only doable right after a resource is created to make the initial ownership much more explicit. Since simply using the resource has the same effect, in the real world this is merely a technicality in the specification.
Small details and caveats
1.) Specialized vs crude barriers
The specialized buffer and image memory barriers were invented for older desktop cards (e.g. GCN based AMD GPUs), ones that had to know when they could flush caches into global memory or had to decompress images before use. For the most part this no longer applies to modern cards and drivers generally throw away most of the information we supply to them through these barriers. For this reason, when targeting modern hardware, crude memory barriers can be just as efficient as them and require less CPU time to manage. This also makes it easier to apply the techniques to a bindless renderer.
2.) Usefulness of image layouts
Similarly to the previous point, image layouts were created for a time, when
textures couldn’t be decompressed on the fly, so you would need to allocate time
for that operation. Modern GPUs can handle this dynamically, so most memory
layout transitions are not necessary anymore and it’s just as fast to keep
images in the GENERAL layout as always using the most optimal one. One
exception to this of course is any time we’re accessing the texture in an
external system (e.g. presentation).
In theory whether GENERAL is just as fast as the optimal layouts or not can be
queried by looking for the VK_KHR_unified_image_layouts extension, support for
it guarantees that this is the case, but the opposite isn’t necessarily true as
the extension is fairly new at the time of writing this. As with everything in
Vulkan, the best way to know for sure whether you need to use a specific type of
barrier or not is to profile it on your target hardware.
3.) Synchronization scope precision
While we can create barriers that block execution between any two pipeline
stages, the specification doesn’t guarantee that the driver will be able to
match this precision. For instance, it’s not hard to imagine a driver that can’t
block a draw command between the LATE_FRAGMENT_TESTS and
COLOR_ATTACHMENT_OUTPUT stages as this would require caching the outputs
somewhere. In practice when a source stage mask is specified, the driver is
allowed to place the edges of the actual synchronization scope anywhere after
that point if necessary and similarly for the destination stage mask, the
synchronization scope can include any stages before the specified ones.
Architecting barriers from scratch
Because of the reasons mentioned in the previous chapter, unless you have a
specific reason to, I recommend only using global memory barriers and image
memory barriers and transitioning images to the GENERAL layout when necessary.
By far the easiest way to handle this process is to disregard synchronization entirely when originally writing new rendering code and just turn on the synchronization validation messages. These usually tell you both what kind of hazard you’re dealing with and what stages and access types the resources should be synchronized for.
If this is not an option (e.g. because you’re working on an automated system), then barrier creation essentially becomes a 4 step process:
- Keep track of when resources are written to or read from and within what pipeline stages and with what access types these operations happen.
- Split up your commands into groups in a way where within a single group a resource can only be read from by any arbitrary amount of commands or be written to by at most one command.
- Identify the hazards between the groups. If a group is reading from a resource, then the last group that wrote to the same resource will cause a RAW hazard, same for WAW and WAR types.
- For every hazard, insert a pipeline barrier at the appropriate time (somewhere between the groups that are affected by it). In the source pipeline stage and access masks include all pipeline stages and access types from the first group, in the destination stage and access masks include the stages and accesses from the second group.
Example
Let’s look at the following example: A, B, C and D are resources accessed in the following commands:
- (1) reads from A and writes to B.
- (2) reads from A and writes to C.
- (3) reads from C and writes to D.
- (4) reads from B and writes to C.
- (5) writes to D.
- (6) reads from A and B and writes to C.
- (7) writes to A.
We already finished step 1, now onto 2. Easiest approach to this one is a greedy algorithm that adds commands to the current group until it can’t anymore due to an incompatible access, in which case it starts a new one.
- First group starts out empty. The reads and writes of (1) don’t interfere with any reads and writes in the first group, so we can include it.
- Reading from the same resource multiple times is allowed, so (2) can be included in the first group.
- We wrote to C in the first group, so (3) can’t be included in it, we close the first group and start a new one that we place (3) into immediately.
- (4) is trying to write to C, but the second group already read from it, so we must start a new group again for (4).
- (5) doesn’t share any resources with the current group, so we can include it.
- The current group read from B and wrote to C, so (6) must go to a new group once again.
- The current group read from A, but (7) is writing to it, so we need to start a new group for it as well.
With that finished, the 5 groups will be:
- [(1), (2)]: reads from A, writes to B and C
- [(3)]: reads from C, writes to D
- [(4), (5)]: reads from B, writes to C and D
- [(6)]: reads from A and B, writes to C
- [(7)]: writes to A
And now we can identify the hazards. Once again the simplest option is to go through the groups one-by-one, looking through all resource operations and comparing them to any unsynchronized writes from before them.
- The first group trivially doesn’t need any synchronization (this might not be the case in a real system where you run multiple frames).
- Between the first and second groups there’s a RAW hazard on C.
- The third group reads from B, which was last written to by the first group (RAW), it writes to C, which was read from in the second group (WAR) and lastly it writes to D which was last written to in the second group (WAW).
- The fourth group reads from A and B, but neither of those have unsynchronized writes (the hazard between the first and third groups already takes care of B). It also writes to C, which creates a WAW hazard with the third group.
- The fifth group writes to A, which was read from in the fourth group (WAR).
So we’ll need to insert the following pipeline barriers:
- A memory barrier between the first and second groups.
- A memory barrier between the first and third groups.
- A memory barrier between the second and third groups.
- A memory barrier between the third and fourth groups.
- An execution barrier between the fourth and fifth groups.