Most of the lunar surface area has been observed from different viewing conditions thanks to the on-orbit work of lunar orbiters, a large amount of images are available for photogrammetric three-dimensional mapping, which is an important issue for lunar exploration. Theoretically, multi-view images contain more information than a single stereo pair and can get better 3D mapping results. In this paper, the semi-global matching method is applied to the object space, and the steps of cost calculation, cost aggregation, and elevation calculation are performed to obtain the three-dimensional coordinates directly. Compared with the traditional image-based semi-global matching method, the object-based semi-global method is more easily extended to multi-view images, which is beneficial for applying multi-view image information. In addition, it does not require steps such as stereo rectification and forward intersection, that is, the overall pipeline is more elegant. Using the LRO NAC images covering Apollo 11 landing area as the experimental data, the result shows that the object-based semi-global matching is competent for the multi-view image matching and the multi-view image result achieves higher accuracy and more details than the single stereo pair. Furthermore, the experimental results of Zhinyu crater data show that this method can also alleviate the uncertainty of the lunar orbiter's positioning to some extent.