Normalization Layers¶

Estimated learning time: 15 minutes

Learning objectives:

• Understand the basics of normalization layers: batch normalization, layer normalization, instance normalization, group normalization.
• Understand the difference between these normalization layers.
• Know how to use these normalization layers in practice. Compare the results of using different normalization layers.

Resources:

• RMSNorm - a better normalization layer