Processing library for feature detection and tracking

The KLT (Kanade-Lucas-Tomasi) is an algorithm for computer vision that tracks points in a sequence of images (a video for example). Points are selected by being at the corner pixels of the image (pixels where edges intersect). Depending on how an image in the sequence distorts into the next, the algorithm computes an optical flow that allows to track the detected points in time, until the disappear from the image and are replaced by new ones.

Christopher Zach at the University of North Carolina at Chapel Hill developed a GPU-implementation of the KLT algorithm, available here, written in C++ and Cg. The main advantage of using the GPU for the computations is a greater performance when compared with CPU implementations, like the one that comes with OpenCV.

I ported Christopher’s GPU-KLT code into Java, so now it can be used as a Processing library called proGPUKLT. The computations remain executed on the GPU, so it should be comparable in performance to the original C++ library. For the moment it works only on NVidia Geforce video cards (6×00 or newer), and requires the CG toolkit to be installed on the computer.

This library has a simple interface to use the KLT tracker:

import codeanticode.progpuklt.*;
FeatureTracker tracker;
PImage img;
tracker = new FeatureTracker(this, 200, 200, 100);
tracker.track(img);
tracker.drawTracks();
tracker.drawFeatures(10);


The library package can be downloaded from sourceforge, and it includes the API documentation and a couple of examples.