9 — Synthesizing Deep Neural Network Architectures using Biological Synaptic Strength Distributions

This paper investigates how synapse-strength affects the efficacy of convolutional neural nets. It’s hypothesized that synapse-strength distribution in different parts of cortex follow different distributions; in lieu of “fully training” a CNN, the authors simply model the synaptic weights of their deep net on a distribution found in biological cortex, and find preliminary results to demonstrate comparable performance in some small-dataset cases: MNIST performance in a deep net using biological distribution weights is 72.52, vs 79.01 in a “fully trained” model.

I’m suspicious of the size of the datasets on which the experimental nets were trained, but the paper notes that this very shortcoming itself may hold some value in cases were minimal training data are available.

I’m looking forward to hearing where this team heads next with this research, and if it holds water in “in-the-wild” work.

PS: By far my favorite part of this paper is the anachronistic citation of Hubel & Wiesel’s seminal 1968 “Receptive Fields” paper:

A particular type of DNN that has proven to be very effective in recent year are convolutional neural networks (CNNs) (see Hubel & Wiesel, 1968).