The use of testosterone replacement therapy (TRT) for hypogonadism continues to rise, particularly in younger men who may wish to remain fertile. Concurrently, awareness of a more pervasive use of anabolic-androgenic steroids (AAS) within the general population has been appreciated. Both TRT and AAS can suppress the hypothalamic-pituitary-gonadal (HPG) axis resulting in diminution of spermatogenesis. Therefore, it is important that clinicians recognize previous TRT or AAS use in patients presenting for infertility treatment. Cessation of TRT or AAS use may result in spontaneous recovery of normal spermatogenesis in a reasonable number of patients if allowed sufficient time for recovery. However, some patients may not recover normal spermatogenesis or tolerate waiting for spontaneous recovery. In such cases, clinicians must be aware of the pathophysiologic derangements of the HPG axis related to TRT or AAS use and the pharmacologic agents available to reverse them. The available agents include injectable gonadotropins, selective estrogen receptor modulators, and aromatase inhibitors, but their off-label use is poorly described in the literature, potentially creating a knowledge gap for the clinician. Reviewing their use clinically for the treatment of hypogonadotropic hypogonadism and other HPG axis abnormalities can familiarize the clinician with the manner in which they can be used to recover spermatogenesis after TRT or AAS use.