The virtual restoration was one of the important protection approaches for the Great Wall. With the help of this type of technology, the damaged Great Wall can be restored to the previous state of the Great Wall at a specific time node so that the tourists can visit the Great Wall before the damaged in the computer. This this paper presented a virtual restoration framework for the “Moon Gate” located on the XinGuangWu Great Wall so that virtual recovered results can guide the researcher to repair the Great Wall in the real world without the secondary damage. This method can be divided into 4 parts: (1) collection of evidence based on the modified scale of evidence; (2) the fusion of DIM point clouds; (3) excavation of the reconstruction technology from the collected evidence; (4) the generation of 3D model. The experimental results showed that the maximum error of the reconstructed “Moon Gate” was 1.088 cm without considering the brick seam and our proposed virtual restoration framework of the Great Wall was robust and accuracy, and it has certain reference significance for guiding the restoration of the damaged Great Wall.