01 28th January, Sunday

Sunday, 28th January

---

I have continued to work with the code in the following git repo: https://github.com/csong27/ml-model-remember based on the paper here https://arxiv.org/abs/1709.07886

I've specifically been aiming to recreate figure 6 of the paper, as a starting point to my analysis of the weight distributions within neural networks. The code here uses the lasagne library, something with which I have had no previous experience, it might be worth me asking around the department to see if there is anyone who would be able to give me some pointers using this framework (and to verify my reasoning about what I have done up to now), but so far I have managed to make progress.

I can produce a histogram of the weights of the final layer of the network, but at the moment I have only been able to train the network for a single epoch on the smallest version of resnet possible (78042 parameters, 18 mins on my laptop, validation accuracy 62%), although the distributions do seem to be slightly different when the model is trained with and without attacks.

The network they used is based on resnet, http://arxiv.org/abs/1512.03385 (paper), https://github.com/Lasagne/Recipes/blob/master/papers/deep_residual_learning/Deep_Residual_Learning_CIFAR-10.py (github).

The next step I am currently looking towards is so expand what I have already done to the same reduced size network with slightly more epochs and to gather parameter information from all of the layers. At the moment I will focus on final distributions but I could potentially analyse their change over time. I'm hoping that with more data I will at least be able to spot the attacks given in the paper above.