209 — idtracker.ai: Tracking all individuals in large collectives of unmarked animals

idtracker.ai is a system that tracks animals’ positions over time.

Why is this unique or important?

One extremely common issue in animal-tracking — a protocol used in quantifying animal behavior — is that of animal identification. It’s easy to keep track of animals with distinct features, but…how do you keep track of dozens of visually almost-indistinguishable individuals moving in multiple dimension? This problem is so pervasive that it’s not uncommon to paint a dot on individuals before tracking their movement over time.

But this isn’t possible all the time, for obvious reasons. In particular, tracking small organisms, or organisms that move in 3D (e.g swimming or crawling) are very difficult to track using conventional off-the-shelf mechanisms.

The tracker works through a series of stacked neural networks: The first net determines when individuals are crossing paths or touching. The second system assigns identities to each individual and then projects their trajectories throughout the duration of the video.

This identification system relies on the fact that individuals spend more time not crossing or touching than they spend in touching or crossing configurations; if this is not the case, the system includes extra protocols to aid in identification training.

This software is open-source (woo!). Hopefully, I’ll have an example use of this system for you to look at someday soon!