Author's profile photo

Aylwin's tech blog

Sprinkle a bit of curiosity in everything I do 🧪

Calculating parameters and computational cost in one convolutional layer

2021-10-25

Overview of Convolutional Operation

How many parameters?

Suppose we have 2 filters that are 3x3x3 in one layer of the neural network. How many parameters does the layer have?

  • Convolution operation has the advantage of fixing the number of parameters regardless of the size of the input image.
  • The number of filters decide the depth of the output dimension. For example, we can have 10 filters to detect specific colour, vertical edge, horizontal edge etc.
  • The number of learnable parameters are:
    = The number of filter weights + bias
    = (filter width x filter height x filter depth x # filters) + (number of filters)
    = (3x3x3x2) + (2)
    = 56 (learnable parameters)

What is the computational cost in this layer?

Calculating Computational Cost based on Output Activation

  • The output dimension is 4x4x2
  • For every output pixel, we have to compute 3x3x3 operations.

  • Computational cost:

    = (# Filter params) x (# filter positions) x (# of filters)

    = (3x3x3) x (4x4) x (2)

    = 864 (operations)