Sakana AI has introduced PC-ALM (Augmented Lagrangian Predictive Coding), a local alternative to backpropagation designed to train deep neural networks using only layer-local dynamics.

Standard deep learning relies on backpropagation, which requires a synchronized forward and backward pass. In contrast, PC-ALM replaces these passes with layer-local dynamical systems where each layer communicates only with its neighbors. By incorporating Lagrange multipliers, the method allows each layer to act as a PI (proportional-integral) feedback controller, enabling the distribution of supervision credit across the network.

In experimental evaluations, PC-ALM successfully trained residual MLPs up to 1000 layers on datasets such as MNIST and Fashion-MNIST, achieving performance nearly matching standard backpropagation. While traditional predictive coding (PC) often suffers from signal decay in deep and narrow networks, PC-ALM maintains effective credit propagation without increasing the inference cost. This research aims to bridge the gap between artificial deep learning and the biologically plausible, distributed credit assignment observed in the brain.

Sources: