In this article we present an algorithm for the fusion of depth images derived by dense image matching (DIM). One key idea of our algorithm is to generate a 2D triangulation for each available depth map in the image sequence using a restricted quadtrees (RQT). On the one hand this guarantees <i>matching triangulations</i>, on the other hand this creates the possibility to reduce points in the noise range not contributing to the geometry in a controlled manner. By vertex decimation computational efforts in subsequent processing steps are eased. In order to reduce IO overhead, the algorithm is designed in an iterative way: an initial triangulation is lifted to 3D space and, if pixel footprints are comparable, updated using depths of the subsequent map in the sequence. Previously not observed surface regions or surface patches observed only with adverse precision are removed from the existing model and updated by more appropriate triangulations. Thereby differences in scale across depth maps are handled which is particularly important to preserve details and obtain surfaces with the best reconstruction geometry. To remove outliers visibility constraints are forced. The input is overlapping depth images and their poses in space, the output are point coordinates representing the surface, their respective normals and to some degree spatial neighbourhood information of points represented as a non-watertight mesh. The performance of the algorithm will be evaluated on a close range and a oblique aerial dataset.