The use of an acoustic nonlinear response has been accepted as a promising alternative for the assessment of micro-structural damage in metallic solids. However, the full mechanism of the acoustic nonlinear response caused by the material micro-damages is quite complex and not yet well understood. In this paper, the effect of material microstructural evolution on acoustic nonlinear response of ultrasonic waves is investigated in rolled copper and brass. Microstructural evolution in the specimens is artificially controlled by cold rolling and annealing treatments. The correlations of acoustic nonlinear responses of ultrasonic waves in the specimens corresponding to the microstructural changes are obtained experimentally. To eliminate the influence of attenuation, which was induced by microstructural changes in specimens, experimentally-measured nonlinear parameters are corrected by an attenuation correction term. An obvious decrease of nonlinearity with the increase of grain size is found in the study. In addition, the influences of material micro-damages introduced by cold rolling on the acoustic nonlinear response in specimens are compared with the ones of grain boundaries controlled by heat treatment in specimens. The experimental results show that the degradation of material mechanical properties is not always accompanied by the increase of acoustic nonlinearity generated. It suggested that the nonlinear ultrasonic technique can be used to effectively characterize the material degradations, under the condition that the variations of grain sizes in the specimens under different damage states are negligible.