FLOPs of Convolution

In this article, I will introduce the concept of FLOPs (Floating Point Operations Per Second) as it pertains to the convolution operations in Deep Learning.

To begin, FLOPs stands for the number of floating-point multiplication and addition operations. Essentially, it measures how many times these operations are executed.

Let’s delve into the FLOPs of convolution using a diagram.

The following diagram illustrates the FLOPs for a single convolution operation

This operation comprises multiplication for each element of the kernel (highlighted in blue) and the summation of operations where multiplied elements are combined (indicated in green).

Given that the aforementioned operation recurs based on the number of input image channels, the output dimensions, and the number of output channels, we can calculate the FLOPs for the convolution as follows:

Below is the corresponding Python code:

W_in = 100
H_in = 100
C_in = 1
C_out = 1
K_width = 3
K_height = 3
S = 1

W_out = (W_in - K_width) // S + 1
H_out = (H_in - K_height) // S + 1

print("W_out:", W_out, "H_out:", H_out)

FLOPs_per_operation = (K_width * K_height + (K_width * K_height - 1)) * C_in

Total_FLOPs = FLOPs_per_operation * W_out * H_out * C_out
print("Total_FLOPs:",Total_FLOPs)