84 — Worm-level Control through Search-based Reinforcement Learning

Lechner et al have studied the tap-withdrawal “TW” reflex in C. elegans, learning about the simple circuitry responsible for avoiding touches on either the nose or tail of the worm. (When the worm senses a mechanical touch, it will move away from that object.)

The circuit responsible for TW is only 11 neurons, with chemical and electrical synapses between them. Because the grammar of the worm’s response to touch is so similar to the “inverted pendulum” system (moving away from stimulus is similar to the task of balancing the pendulum shape), it is possible to “learn” the synapse directions and weights (or at least one possible configuration of them) by using the TW circuit to control the inverted pendulum.

The researchers had success with this model when they added assumptions about certain variables to bound the search space. For example, they declared that capacitance must be between 1mF and 1F; reversal potential had to be 0 for excitatory and -90mV for inhibitory; etc.

The online demo video is pretty exciting (but very short). You can watch the worm circuit control the inverted pendulum system here.