Batch Normalization Reproduction
Python - PyTorch - NumPy - Matplotlib
Reproduction of Ioffe & Szegedy's ICML 2015 Batch Normalization paper, validating its core claims on MNIST and CIFAR-10.
Reproduced two central findings from Ioffe & Szegedy's "Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift" (ICML 2015) as part of ECE 447 at the University of Alberta.
Built a 3-layer fully connected sigmoid network on MNIST to reproduce Figure 1(a), demonstrating that BN improves test accuracy from ~73-83% to ~97-98% across varying network widths.
Designed a 6-layer CNN on CIFAR-10 to validate BN's learning rate stability claim, showing non-BN networks diverge (NaN loss) at LR = 0.1 while BN networks train stably to ~80% accuracy.
Implemented original ablation studies on hidden layer width and optimizer choice (SGD vs. Adam) to extend the paper's findings beyond its original setup.