Label Smoothing and Adversarial Robustness

Author

Arpit Aggarwal

Introduction to the Project

This project surveys the various adversarial attacks on neural networks. This is useful in cases where we deploy the deep learning systems in real-world scenarios like autonomous driving. First, different CNN architectures(VGG-16, VGG-19 and ResNet-50) are trained on CIFAR-10 dataset. Then after training these architectures, various adversarial examples from CIFAR-10 dataset are generated and consequently these examples are used to evaluate how robust these pretrained models are to various forms of adversarial attacks. Different adversarial attacks like FGSM, I-FGSM and DeepFool were implemented and their consequent adversarial example was reported where the pretrained model mis-classified the input.

Data

The data used for this task was CIFAR-10 dataset. The dataset has been divided into two sets: Training data and Validation data. The analysis of different CNN architectures for image classifcation on CIFAR-10 dataset was done on comparing the Training Accuracy and Validation Accuracy values.

FGSM Attack

FGSM is a non-targeted white box attack. The aim of the attack is to ensure the input image is perturbed with noise which makes it to mis-classify the input other than the actual target class. This is done by adding the gradient of the loss function(predicted output and actual class) wrt to the input image to the input image so that it leads to classifying the input to a class other than the actual target class. An example is shown below:

I-FGSM Attack

I-FGSM is a targeted white box attack. The aim of the attack is to add noise in the input image so that it can classify the input image to a specific targeted class. This is done by adding the gradient of loss function(predicted output and targeted class rather than the actual class) wrt to the input image to the input image so that it leads to classifying the input to a specific targeted class. An example is shown below:

Results

Accuracy vs Smoothing Factor(VGG-16 trained on CIFAR-10)

Accuracy vs Smoothing Factor(ResNet-50 trained on CIFAR-10)

Training-Evaluation Results

The results after using different CNN architectures on CIFAR-10 dataset are given below:

1. ResNet-50(smoothing factor=0.0, epochs=300, lr=0.005, wd=0.001)
   

Validation Accuracy = 89.14%

1. ResNet-50(smoothing factor=0.1, epochs=300, lr=0.005, wd=0.001)
   

Validation Accuracy = 89.00%

1. ResNet-50(smoothing factor=0.3, epochs=300, lr=0.005, wd=0.001)
   

Validation Accuracy = 88.03%

1. ResNet-50(smoothing factor=0.5, epochs=300, lr=0.005, wd=0.001)
   

Validation Accuracy = 87.62%

1. ResNet-50(smoothing factor=0.7, epochs=300, lr=0.005, wd=0.001)
   

Validation Accuracy = 86.37%

1. ResNet-50(smoothing factor=0.9, epochs=300, lr=0.005, wd=0.001)
   

Validation Accuracy = 10.59%

1. VGG-16(pretrained on ImageNet, smoothing factor=0.0, epochs=250, lr=0.001, wd=5e-4)
   

Validation Accuracy = 89.01%

1. VGG-16(pretrained on ImageNet, smoothing factor=0.1, epochs=250, lr=0.001, wd=5e-4)
   

Validation Accuracy = 89.10%

1. VGG-16(pretrained on ImageNet, smoothing factor=0.3, epochs=250, lr=0.001, wd=5e-4)
   

Validation Accuracy = 88.74%

1. VGG-16(pretrained on ImageNet, smoothing factor=0.5, epochs=250, lr=0.001, wd=5e-4)
   

Validation Accuracy = 88.45%

1. VGG-16(pretrained on ImageNet, smoothing factor=0.7, epochs=250, lr=0.001, wd=5e-4)
   

Validation Accuracy = 87.83%

1. VGG-16(pretrained on ImageNet, smoothing factor=0.9, epochs=250, lr=0.001, wd=5e-4)
   

Validation Accuracy = 10.54%

Software Required

To run the jupyter notebooks, use Python 3. Standard libraries like Numpy and PyTorch are used.

Credits

The following links were helpful for this project:

1. https://pytorch.org/tutorials/beginner/fgsm_tutorial.html
2. https://www.youtube.com/channel/UC88RC_4egFjV9jfjBHwDuvg
3. https://github.com/pytorch/tutorials