06 2nd February, Friday

I've been attempting to implement the weight point / image point visualisation in t-SNE but it turns out that the version of t-SNE that sklearn uses doesn't support transforming new points via an existing transformation. That is if I transform the image points I cannot then transform the weight points in the same way after. I have tried transforming them all together, but the result has just been that the weights form their own cluster in the centre with other clusterings around. Interestingly, some of the weight points seem to be clustering a little and others seem to be just as spread out as in the first epoch. It appears that within this weight clustering some of the weights are moving and some are not, which is not as I would have expected. Perhaps these weights that aren't moving correspond to the weights in the histogram that are lower?

I am next going to try just mapping weights across and labelling with the magnitude of the vector to investigate this further.

Update: I've mapped the weight vectors on their own into 2D, labelling with vector magnitudes. It seems that their is no distinguishable clustering, but vectors with higher magnitudes are pushed outwards and vectors with lower magnitudes are clustered together nearer the middle. Although interesting, I think I need a more sensible way of interpreting weights and I need to investigate further to find a way of tying together the information I have for the weights and the activations. I'll be looking next to setting up a minimal example in the hope that it will allow me to investigate more thoroughly.