Lucas-Kanade Optical Flow

Original article was published on Artificial Intelligence on Medium

Lucas-Kanade Optical Flow

This is part of my series on GPS Denied Navigation

Here is an LK optical flow determining the x and y velocity of the aircraft. The code to generate this is at the end of this post.

The code to generate this is below

In Part 2 we discovered the following equation describes a tracked pixel with unknown velocities u and v as spacial motion induced by the camera.

There are two unknowns and a single equation which means this is really tough to solve. To get over this, instead of looking a single pixel’s movement over time, we want to track a collection of colocated pixels assuming they have the same u and v. Now we can capture multiple equations and solve for u andv with least-squares.

We solve the least-squares problem with the -b + A x = 0 method

When eigenvalues are small, there is no gradient in all directions, it is likely we are looking at pixels over a flat area that don’t show any correlation, like the river in the image above. When eigenvalues show a large ratio, 1 strong, 1 weak gradient, it means we’re looking at an edge, eg the riverbank. This collection is only useful for determining either u or v but not both since we can clearly see when we move laterally, but if we move up the edge, from the camera’s point-of-view we might not be moving at all.

Where we find large eigenvalues, thats where we know it is a highly textured region of the image that serves as a good collection of pixels to track.

Similar to the Shi-Tomasi feature detector we used in Part 1, the Optical Flow algorithm has many tunable paramters. We’ll be using cv.calcOpticalFlowPyrLK which uses the Lucas-Kanade method.