Original article was published on Artificial Intelligence on Medium
Lucas-Kanade Optical Flow
This is part of my series on GPS Denied Navigation
Here is an LK optical flow determining the x and y velocity of the aircraft. The code to generate this is at the end of this post.
In Part 2 we discovered the following equation describes a tracked pixel with unknown velocities
v as spacial motion induced by the camera.
There are two unknowns and a single equation which means this is really tough to solve. To get over this, instead of looking a single pixel’s movement over time, we want to track a collection of colocated pixels assuming they have the same
v. Now we can capture multiple equations and solve for
v with least-squares.
When eigenvalues are small, there is no gradient in all directions, it is likely we are looking at pixels over a flat area that don’t show any correlation, like the river in the image above. When eigenvalues show a large ratio, 1 strong, 1 weak gradient, it means we’re looking at an edge, eg the riverbank. This collection is only useful for determining either
v but not both since we can clearly see when we move laterally, but if we move up the edge, from the camera’s point-of-view we might not be moving at all.
Where we find large eigenvalues, thats where we know it is a highly textured region of the image that serves as a good collection of pixels to track.
Similar to the Shi-Tomasi feature detector we used in Part 1, the Optical Flow algorithm has many tunable paramters. We’ll be using cv.calcOpticalFlowPyrLK which uses the Lucas-Kanade method.