15 — Large-Scale Simulations of Plastic Neural Networks on Neuromorphic Hardware

The SpiNNaker team designed the ARM processors of the simulator to tesselate hexagonally (like a beehive), in order to maximize the speed with which signals can be communicated across what the authors call the “network fabric”, which is a very badass name for a neuromimetic computer.

The purpose of this paper is to demonstrate SpiNNaker’s ability to simulate BCPNNs (Bayesian Confidence Propagation — a set of equations that act as a scaffold on which to build neuroplasticity models), and its ability to generalize to simulate arbitrary neural networks.

An important distinction between computational neural networks and biological ones is the importance of time on the signals of a biological network: There are no “epochs” in a brain, or “simulation steps”, and there are no hidden layers. Computational neural networks are very much linear — and if convolution takes place, it generally happens after a layer is fully processed. In a biological brain, different sized neurons take different amounts of time to communicate, on a continuous scale. If you wait for all signals to propagate before processing the next simulation step, you’ve thrown away very meaningful data. (In fact, the circuit responsible for telling us from where in space a sound originates relies extensively on the fact that some signals are processed microseconds earlier from one ear than the other.)

That SpiNNaker models itself so closely on biological neurons means that it is able to simulate millions or tens of millions of plastic (that is, dynamic) synapses at 0.5× realtime, using a tiny fraction of the amount of energy that would be needed to run the equivalent simulation on conventional supercomputer hardware.

Still a far cry from the handful of watts of energy used by the human brain — but we’re closer than ever before.