New AdamMAD Optimizer Swaps Squared Gradients for Mean Absolute Deviation to Cut Training Oscillations
Researchers report that replacing Adam's squared-gradient second moment with the mean absolute deviation yields smoother training, fewer oscillations and faster ...
