To summarize : the input and skip connection are used to decide what parts of the skip connection to focus on.
The attention operations allow for a selective picking of the information contained in the values. This attention distribution is extracted from what is called the query (the input) and the values (the skip connection). The attention distribution is multiplied by the skip connection feature map to only keep the important parts. It allows for the direct connection to focus on a particular part of the input, rather than feeding in every feature. This way, when “reconstructing” the mask of the image, the network learns to use these features, because the features of the contracting path are concatenated with those of the expanding path.Īpplying an attention block before this concatenation allows for the network to put more weight on the features of the skip connection that will be relevant. What is interesting about the UNet is the fact that the skip connections allows for features extracted by the encoder to be used directly during the decoder. In a UNet, the contracting path can be seen as an encoder and the expanding path as a decoder. this attention map is then multiplied by the skip input to produce the final output of this attention block.another 1x1 convolution and a sigmoid, to flatten to a single channel with a 0-to-1 score of how much importance to give to each part of the map.
after an upsampling operation (to have the same size), they are summed and passed through a ReLU.both x and g are fed into 1x1 convolutions, to bring them to the same number of channels, without changing the size.Source : Attention UNet: learning where to look for the Pancreas The dimensions here assume a 3-dimensional input image. What changes is the expanding path, and more precisely, the attention mechanism is integrated into the skip connections.īlock diagram of the attention block. The architecture uses the standard UNet as a backbone, and the contracting path is not changed. If you want a refresher of how a standard UNet works, this article does a perfect job. In this paper, the authors propose a way to apply the attention mechanism to a standard UNet. UNet is the go-to architecture for segmentation and most of the current advances in segmentation use this architecture as a backbone.
#JORDAN 1 ATTENTION ATTENTION CODE#
Note: of course, both the code and explanations are simplifications of the complex architectures described in the papers, the aim is mostly to give an intuition and a good idea of what is done and not to explain every detail.
0 kommentar(er)
