Sakana AI’s Error Diffusion Trains Dale-Compliant Dual-Stream Networks, Reaching 96.7% MNIST and 61.7% CIFAR-10 Without Backpropagation

Sakana AI’s Diffusing Blame paper introduces Error Diffusion (ED), a local learning rule that trains Dale-compliant dual-stream networks without weight transport, achieving 96.7% on MNIST and 61.7% on CIFAR-10. The architecture splits each layer into excitatory and inhibitory streams, using four non-negative weight matrices and modulo error routing. ED also outperforms backpropagation-free methods in reinforcement learning tasks like HalfCheetah.