A 0.4C-2Si-1Cr-1Mo steel with an ultrafine elongated grain (UFEG) structure and an ultrafine equiaxed grain (UFG) structure was fabricated by multipass caliber rolling at 773 K and subsequent annealing at 973 K. A static three-point bending test was conducted at ambient temperature and at 77 K. The strength–toughness balance of the developed steels was markedly better than that of conventionally quenched and tempered steel with a martensitic structure. In particular, the static fracture toughness of the UFEG steel, having a yield strength of 1.86 GPa at ambient temperature, was improved by more than 40 times compared with conventional steel having a yield strength of 1.51 GPa. Furthermore, even at 77 K, the fracture toughness of the UFEG steel was about eight times higher than that of the conventional and UFG steels, despite the high strength of the UFEG steel (2.26 GPa). The UFG steel exhibited brittle fracture behavior at 77 K, as did the conventional steel, and no dimple structure was observed on the fracture surface. Therefore, it is difficult to improve the low-temperature toughness of the UFG steel by grain refinement only. The shape of crystal grains plays an important role in delamination toughening, as do their refinement and orientation.