Achieving an ultra-broadband range is an essential development direction in terahertz techniques; however, a method to cover the full terahertz band by using a highly efficient antireflection (AR) coating that could greatly increase the efficiency of terahertz radiation is still lacking. It is known that structures possessing a graded-index profile can offer a broadband AR effect, and such structures have been widely used, especially in the visible range. In this paper, first, we tuned the refractive index of a cyclo-olefin polymer (COP) by using a TiO2 dopant, and a polymer–TiO2 composite with a refractive index of 3.1 was achieved. We then fabricated a surface-relief structure with a graded-index profile by using a hot-embossing method. The structure on the silicon substrate can provide an excellent AR effect, but the working band is still limited by its scale of sag and swell. To obtain an ultra-broadband AR effect, we then proposed a flat six-layer structure; a graded-index profile was obtained by casting epoxy–TiO2 composites in the order of a high index to lower indices. With a very well controlled refractive index and thickness of each layer, we achieved an AR effect of <2% in the ultra-broadband of 0.2–20 THz.