Regarding the attention map

[gemcollector]

I noticed that you drew an attention map in the text. How do you draw an attention map? Can you provide the source code?

[MishaLaskin]

The attention maps are just activations of one of the CNN layers. To make them - pick a CNN layer run a softmax over each channel and take the mean.

…

On Sat, Sep 11, 2021 at 8:53 AM gemcollector ***@***.***> wrote: I noticed that you drew an attention map in the text. How do you draw an attention map? Can you provide the source code? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#16>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABHWQWMZLSO2266RL2EIANTUBNUQDANCNFSM5D3BEZQA> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

[longfeizhang617]

I noticed that you drew an attention map in the text. How do you draw an attention map? Can you provide the source code?

I have noticed the attention map too, have you solved this issue?

[Dongzhou-1996]

I noticed that you drew an attention map in the text. How do you draw an attention map? Can you provide the source code?

I have noticed the attention map too, have you solved this issue?

The author mentioned that their method is similar to the algorithm proposed in PAYING MORE ATTENTION TO ATTENTION:
IMPROVING THE PERFORMANCE OF CONVOLUTIONAL NEURAL NETWORKS VIA ATTENTION TRANSFER, which simply sums the activation tensor of any convolutional layer along the channel dimension (i.e. a tensor in [BxCxWxH] -> attention map [BxWxH]). Corresponding codes can be seen in this link. The only difference between these two methods is the spatial softmax operation, I think it is added by Laskin to show the attention map better and highlight the fixation of a convolutional layer.