Visualizing Feature Maps and CNN Filters
Visualizing learned filters and activation feature maps provides insights into the spatial features captured by CNN layers, opening the "black box" of neural representations.
Visualizing Learned Filters
Inspecting the learnable weights of convolutional layers reveals the specific visual patterns the network is tuned to detect.
Inspecting Weight Tensors
In convolutional layers, weights are 4D tensors of shape \\((C_{out}, C_{in}, K_H, K_W)\\). Early layers learn simple spatial filters that detect edges, shapes, and color transitions. By extracting the weights of the first layer, we can plot them as images to visualize what patterns they respond to.
Because the first layer is connected directly to the input RGB image, its filters are easy to interpret. They often resemble classical edge detectors like Gabor filters. Plotting these weights allows developers to verify that the network is learning general spatial priors rather than memorizing random training noise.
Deeper Layer Interpretation
Visualizing filters in deeper layers is more challenging because they do not process the raw input pixels directly. Instead, they operate on the feature maps of previous layers. To visualize what deep filters detect, researchers use techniques like Activation Maximization, which performs gradient ascent on a random input image to maximize the activation of a specific neuron.
This optimization process generates an image that represents the pattern the neuron has learned to recognize. As we move deeper, these generated images evolve from simple textures to complex semantic shapes, showing the hierarchical composition of features inside the network.
Visualizing Feature Maps (Activations)
Feature maps track the activations of convolutional layers, showing which regions of an image trigger specific filters.
Extracting Intermediate Activations
A feature map is the output of a convolutional layer after passing an input image through it. By extracting these intermediate feature maps during the forward pass, we can see which parts of the input image activate specific channels.
Early feature maps preserve high-resolution spatial details, showing exactly where edges and corners are located. Deep feature maps are low-resolution and highlight abstract concepts, such as where an object's face or wheels are situated, demonstrating how spatial downsampling extracts semantic details.
Registering Forward Hooks in PyTorch
PyTorch provides a mechanism called "hooks" to extract intermediate activations without modifying the model's forward method. A forward hook is a function that is executed automatically during the forward pass, allowing us to copy and store the activation tensors.
<pre><code class="language-python">import torch import torch.nn as nn class SimpleModel(nn.Module): def __init__(self): super().__init__() self.conv1 = nn.Conv2d(3, 16, kernel_size=3, padding=1) self.pool = nn.MaxPool2d(2, 2) def forward(self, x): return self.pool(self.conv1(x)) # Hook function definition activation = {} def get_activation(name): def hook(model, input, output): activation[name] = output.detach() return hook model = SimpleModel() # Register hook on conv1 model.conv1.register_forward_hook(get_activation('conv1')) x = torch.randn(1, 3, 32, 32) out = model(x) print("Hooked activation shape:", activation['conv1'].shape) # [1, 16, 32, 32]</pre>The forward hook captures the output tensor during the computation graph execution. Calling detach() on the output tensor is a critical step, as it prevents the hook from retaining gradients in memory, which would cause memory leaks during training.