A coupled vibration model of hot rolling mill rolls under multiple nonlinear effects is established by considering the nonlinear spring force produced by the hydraulic cylinder, the nonlinear friction between the work rolls, the dynamic variation of rolling force, and the effect of external excitation as well as according to the structural constraints of a four-high hot rolling mill in the vertical and horizontal directions. The amplitude-frequency response equation of rolling mill rolls is determined by using a multiple-scale approximation method. Furthermore, use of actual data for simulation indicates that the internal resonance is the main cause of coupling vibration of the rolling mill rolls. In addition, changes in the movement displacement of the hydraulic cylinder and the coupling parameters strongly affect the coupling system of the rolling mill rolls. Finally, the study of the dynamic bifurcation characteristics of the rolling mill rolls indicates that, with varying external excitation amplitude, the vibration of rolls alternates between periodic motion, period-doubling motion, and chaotic motion in both vertical and horizontal directions. This is one of the reasons for the appearance of periodic light and dark stripes on the strip surface. Furthermore, the range of the external excitation amplitude (F0) at which the rolling mill roll system vibrates violently, that is, 5.68e5 N < F0 < 5.84e5 N and F0 > 6.12e5 N, must be avoided. The research results can provide a theoretical reference for further exploration of the coupling vibration mechanism of hot rolling mills.