In space exploration missions, planetary rovers require precise navigation to ensure operational safety and efficiency. This paper presents a novel visual-LiDAR odometry method designed specifically for planetary rovers, aiming to address the challenges of accurate navigation in planetary environments that are GNSS-denied and characterized by low texture and unstructured surfaces. By combining visual information from a monocular camera with the high-precision geometric measurements of the LiDAR sensor, the system is capable of achieving accurate localization and pose estimation in a planetary-like scenario. Furthermore, the incorporation of ground segmentation for the LiDAR point cloud and the application of ground plane constraints can effectively alleviate the attitude drift during the motion process, thereby enhancing the overall accuracy. Experimental results on the Erfoud dataset show that the proposed system achieves positioning accuracies (i.e., absolute trajectory error) of 3.334% and 3.869% over the full trajectory length of two sequences, outperforming state-of-the-art methods. This research provides a promising solution for the autonomous navigation of planetary rovers in complex environments.