Effects of exposure and gain on FAST detector

Behavior of feature detection on exposure and gain changes

An interesting behavior I’ll likely reference in the future is the effect that exposure and gain have on feature detection, so let’s do a short blog post out of it and show its behavior in the next video.

In this video we can see Basalt using the FAST feature detection algorithm from OpenCV (cv::FAST) on images fed from the OV7251 sensors found in a Samsung Odyssey+ from Monado. Notice how these camera parameters affect the detected features. In particular, notice how increasing gain also increases the number of unuseful features detected by FAST even when the cameras are totally covered. On the other hand, the features we obtain by increasing exposure are a lot more stable. Keep in mind that high values of exposure increase motion blur too, which is a no-go for tracking algorithms.

The auto exposure-gain algorithm found in Monado tries to do a good enough trade off between these two values but even then, a significant amount of incorrect features are detected, in any case Basalt is able to filter them properly and Basalt needs to filter them aft.

EDIT: Here is a quick explanation of how Basalt detects features:

• It subdivides the image in a grid
• It uses the best cv::FAST detected features on each grid cell starting with a maximum threshold (40 by default) and if it doesn’t find any feature it halves it and tries again a couple of time until a minimum acceptable threshold is reached (5 by default).
• The feature is only used if it was observed in at least two keyframes and the triangulation between the keyframes succeeds. This observation will only happen if the feature was successfully tracked through optical flow between two keyframes; either temporally in the same camera or spatially between different cameras.

Hopefully this shows how having more features will likely decrease performance in the frontend (more wrong features to compare against in the keyframe-pair triangulation search) of the system and thus having good images makes everything run faster and more accurate.