I have been learning about weight initialization in deep learning. The notes below capture the methods and activation-function associations I recorded.
I have learnt about weight initialization in deep learning.
One weight-initialization method I noted is Xavier.
I associate Xavier with tanh and sigmoid activation functions.
I associate He with the ReLU activation function.