The model of the observed object plays the key role in the task of object tracking. Models as a set of image parts, in particular, keypoints, is more resistant to the changes in shape, texture, angle of view, because local changes apply only to specific parts of the object. On the other hand, any model requires updating as the appearance of the object changes with respect to the camera. In this paper, we propose a dynamic (time-varying) model, based on a set of keypoints. To update the data this model uses the algorithm of rating keypoints and the decision rule, based on a Function of Rival Similarity (FRiS). As a result, at the test set of image sequences the improvement was achieved on average by 9.3% compared to the original algorithm. On some sequences, the improvement was 16% compared to the original algorithm.