Condition-Based LSTM for Residual Calculation of Optical Flow

Abstract. Accurate state estimation for autonomous unmanned aerial vehicles (UAVs) is essential in GPS-denied environments, such as in indoor tasks. In these situations, Inertial Navigation Systems (INS) perform well in the short term with good accuracy, but they suffer from accumulated error (drift) caused by sensors’ noise and bias. Optical flow (OF) is often utilized to correct this drift by estimating lateral velocity from the motion of images; however, OF is degraded by gyroscope errors when the UAV performs aggressive turns or high-acceleration maneuvers. This paper proposes a lightweight, condition-based Long Short-Term Memory (LSTM) neural network to correct these errors. The LSTM model will identify and correct systematic OF velocity errors when the UAV is performing aggressive maneuvers and it is triggered by applying a hysteresis gate that monitors the magnitude of gyroscope readings. This way, the LSTM is only active in scenarios where OF measurements are less reliable. This approach preserves the efficiency of classical methods and prevents any degradation of system performance during stable flight. Several test scenarios were conducted in the Gazebo simulation environment to evaluate the performance of the algorithm. Overall, the INS/OF/LSTM framework reduces trajectory root mean square error (RMSE) by 63.77 % and velocity RMSE by 28.94 % compared to a typical INS/OF pipeline on average across all test scenarios. This paper demonstrates that condition-based LSTM networks increase the accuracy and reliability of classical optical flow-based velocity estimation used in UAV applications.