In this paper a method to improve the co-registration accuracy of two separate HySpex SWIR and VNIR cameras is proposed. The first step of the presented approach deals with the detection of point features from both scenes using the BRISK feature detector. After matching these features, the match coordinates in the VNIR scene are orthorectified and the resulting ground control points in the SWIR scene are filtered using a sensor-model based RANSAC. This implementation of RANSAC estimates the boresight angles of a scene by iteratively fitting the sensor-model to a subset of the matches. The boresight angles which can be applied to most of the remaining matches are then used to orthorectify the scene. Compared to previously used methods, the main advantages of this approach are the high robustness against outliers and the reduced runtime. The proposed methodology was evaluated using a test data set and it is shown in this work that the use of BRISK for feature detection followed by sensor-model based RANSAC significantly improves the co-registration accuracy of the imagery produced by the two HySpex sensors.