When to use Batch Normalization

TensorFlow
,
1

Hi,
Could someone tell me about when it is recommended to use Batch Normalization? Is it recommended for standard CNNs? Autoencoders for image reconstruction? Excel type data analisys (with a NN)?

Thanks!

1 Like
3

Hi @migueltg, thanks for posting the issue.

It is generally used with CNNs, batch normalization can be used after each convolutional layer and before the activation function.

Similarly, in fully connected networks, it’s common to add batch normalization after the linear transformation and before the activation function of each hidden layer.

However for image reconstruction it is possible that it may hurt performance, especially in the decoder, because it may normalize away important intensity or structural information, leading to blurrier outputs.

You can read more about batch normalization here.

Hope that helps. Thanks!