Traffic signs provide important traffic information for automatic driving, and accurate and complete traffic sign data of HD (High Definition) map provides important data support for intelligent transportation, automatic driving and other emerging service industries. Driving record data fills the data gap of crowd-source updating in HD maps, and the crowd-source updating method of road traffic facilities in HD maps using massive driving record data has become a new research hotspot. In this paper, an incremental HD map traffic sign crowd-source update method is proposed based on the driving record data. The traffic sign detection results are matched with the existing traffic signs in the HD map for traffic sign change detection, and the added results are optimized and fused for position, and the new sign positions are optimized using the unchanged signs to obtain the optimized new traffic sign positions. The experiments in Shanghai show that the matching method can meet the matching requirements of crowd-source updating; the accuracy of the traffic sign positions after position optimization and crowd-source fusion is obviously improved, with an average plane error of 3.69 m and a standard deviation of error of 3.29 m, which can provide data support for crowd-source updating of the HD map.