The ultrasonic guided wave-based structural damage diagnosis method has broad application prospects in different fields. However, some environmental factors such as temperature and loads will significantly affect the monitoring results. In this paper, a reference matching-based temperature compensation for ultrasonic guided wave signals is proposed to eliminate the effect of temperature. Firstly, the guided wave signals measured at different temperatures are used as reference signals to establish the relationship between the features of the reference signals and temperature. Then the matching algorithm based on Gabor function is used to establish the relationship between the amplitude influence coefficient obtained by the reference signal and the corresponding temperature. Finally, through these two relationships, the values of the phase and amplitude influence coefficients of the guided wave signals at other temperatures are obtained in a way of interpolation in order to reconstruct the compensation signals at the temperature. The effect of temperature on the amplitude and phase of the guided wave signal is eliminated. The proposed temperature compensation method is featured such that the compensation performance can be improved by multiple iteration compensation of the residual signal. The ultrasonic guided wave test results at different temperatures show that the first iterative compensation of the proposed method can achieve compensation within the temperature range greater than 7 °C, and the compensation within the temperature range greater than 18 °C can be achieved after three iterations.