Feedforward Network / Multi-layer Perceptron Quiz (Medium)

What is the main advantage of using a deep feed forward network over a shallow one with the same number of parameters?

Which problem arises due to the saturation of the Sigmoid activation function in deep feed forward networks?

Which of the following is an advantage of using the ReLU activation function in feed forward networks over traditional Sigmoid or Tanh?

What is a potential drawback of using ReLU activation in a feed forward neural network?

Which weight initialization technique is particularly recommended for layers with Sigmoid or Tanh activation functions?

When designing a feed forward network for a multi-class classification problem, why is the Softmax function preferred over multiple Sigmoid functions for the output layer?

Skip connections (or residual connections) introduced in architectures like ResNet help alleviate which problem in very deep feed forward networks?

What is a common strategy for weight initialization in an MLP to ensure faster and more stable convergence?

How does the Universal Approximation Theorem relate to MLPs?

In which scenario is the ReLU activation function likely to cause “dying neurons” in an MLP?

For an MLP, which of the following optimizers uses both momentum and adaptive learning rates?