The Vision and Media Lab
Action Recognition of Insects Using Spectral Clustering
We propose a technique to recognize actions of grasshoppers based on spectral clustering. We track the object in 3D and construct features using 3D object movement in segments of video which discriminate between different classes of actions.

We start by tracking the grasshopper using a stereo camera setup. Then employ a background subtraction technique to track the object.We define a set of motion features based upon this tracker output and perform spectral clustering on these W dimensional feature vectors to classify object actions.

We tested our algorithm on 10 minute video of an individual grasshopper.

After clustering the actions of the grasshopper, we used the video synopsis technique of Rav-Acha et al. (CVPR 2006) to create short videos displaying each cluster of similar grasshopper actions. These videos allow one to see in which regions of the cage the grasshopper performed various actions.

Input video

Video synopsis results:

For more information, contact Maryam Moslemi Naeini

Maryam Moslemi Naeini. Clustering And Visualizing Actions Of Humans And Animals Using Motion Features. M.Sc. thesis, Simon Fraser University, 2007.
Maryam Moslemi Naeini, Greg Dutton, Kristina Rothley, Greg Mori. Action Recognition of Insects Using Spectral Clustering. IAPR Conference on Machine Vision Applications, 2007. [pdf]