Polyester fiber and SBR latex cement concrete is prepared as pavement surface material; its fracture properties including fracture toughness, fracture energy, CMOD, and flexural strength are studied comparing with those of normal concrete (NC), polyester fiber modified concrete (FMC), SBR polymer modified concrete (SMC), and the combination of polyester fiber and SBR polymer modified concrete (FSMC). The modification mechanism of the latex and fiber on the concrete was also studied by the methods including X-ray test, chemically combined water, heat of hydration, water loss, and scanning electron microscope. Results indicated that the concrete modified by latex and polyester fiber has flexural strength, fracture toughness, and fracture energy of 44.4%, 397.0%, and 462.8% higher than the reference normal concrete, the polymer retarded the hydration process and reduced the hydration degree of cement at early age, while the hydration degree is promoted by the polymer film for its excellent water resistance after 28 d, and the bond between the fiber and cement paste is improved by the latex.