COURSE OVERVIEW

This practical and highly-interactive course includes real-life case studies and exercises where participants will be engaged in a series of interactive small groups and class workshops.  

This course is designed to provide participants with a detailed and up-to-date overview of Batch Normalization. It covers the vanishing and exploding gradient problems and the effects of poorly scaled inputs on training stability; the normalization techniques and batch normalization as an alternative to input normalization; the mathematics behind batch normalization; the batch normalization in different layers of neural networks; how batch normalization reduces covariate shift and reducing the sensitivity to learning rate selection; the generalization and reducing overfitting; and the batch normalization and activation functions. 

 

Further, the course will also discuss the batch normalization in convolutional neural networks (CNNs) and recurrent neural networks (RNNs) and transformers; the layer normalization versus batch normalization; the instance normalization and group normalization; the difference between batch normalization, weight normalization and the need for batch renormalization in certain applications; handling very small or large batch sizes; and the impact of mini-batch size on normalization statistics. 

During this interactive course, participants will learn the batch normalization in object detection models, generative adversarial networks (GANs) and reinforcement learning (RL); the impact of batch normalization in large-scale datasets including hyperparameter tuning for batch normalization; the evolution of normalization beyond batch normalization and emerging alternatives like normalizer-free networks; and handling batch normalization in real-world inference, converting batch norm layers to fixed scaling and optimizing batch norm for mobile and edge devices .