Nature

A dual enhanced stochastic gradient descent method with dynamic momentum and step size adaptation for improved optimization performance

In modern machine learning, optimization algorithms are crucial; they steer the training process by skillfully navigating through complex, high-dimensional loss landscapes. Among these, stochastic ...
Nature

Fractional order system identification using a joint multi-innovation fractional gradient descent algorithm

This paper proposes a joint multi-innovation fractional gradient descent identification algorithm for fractional order systems. First, the flexibility of fractional calculus is leveraged to design a ...
Semiconductor Engineering

In Situ Backpropagation Strategy That Progressively Updates Neural Network Layers Directly in HW (TU Eindhoven)

A new technical paper titled “Hardware implementation of backpropagation using progressive gradient descent for in situ training of multilayer neural networks” was published by researchers at ...