A B C D E F G H I J K L M N O P Q R S T U V W X Y Z

What does momentum mean in neural networks?

Best Answers

In neural networks, we use gradient descent optimization algorithm to minimize the error function to reach a global minima. read more

A small value of momentum cannot reliably avoid local minima, and can also slow down the training of the system. Momentum also helps in smoothing out the variations, if the gradient keeps changing direction. A right value of momentum can be either learned by hit and trial or through cross-validation. read more

The same concept applies to neural networks, during training the update direction tends to resist change when momentum is added to the update scheme. When the neural net approaches a shallow local minimum it's like applying brakes but not sufficient to instantly affect the update direction and magnitude. read more

Encyclopedia Research

Related Facts

Related Question Categories

Image Answers

What does momentum mean in neural networks? - Quora
Source: quora.com