Internal levels of selected endocrine disruptors (EDs) (i.e., perfluorooctane sulfonate (PFOS), perfluorooctanoic acid (PFOA), di-2-ethylhexyl-phthalate (DEHP), mono-(2-ethylhexyl)-phthalate (MEHP), and bisphenol A (BPA)) were analyzed in blood/serum of infertile and fertile men from metropolitan, urban and rural Italian areas. PFOS and PFOA levels were also evaluated in seminal plasma. In peripheral blood mononuclear cells (PBMCs) of same subjects, gene expression levels of a panel of nuclear receptors (NRs), namely estrogen receptor α (ERα) estrogen receptor β (ERβ), androgen receptor (AR), aryl hydrocarbon receptor (AhR), peroxisome proliferator-activated receptor γ (PPARγ) and pregnane X receptor (PXR) were also assessed. Infertile men from the metropolitan area had significantly higher levels of BPA and gene expression of all NRs, except PPARγ, compared to subjects from other areas. Subjects from urban areas had significantly higher levels of MEHP, whereas subjects from rural area had higher levels of PFOA in both blood and seminal plasma. Interestingly, ERα, ERβ, AR, PXR and AhR expression is directly correlated with BPA and inversely correlated with PFOA serum levels. Our study indicates the relevance of the living environment when investigating the exposure to specific EDs. Moreover, the NRs panel in PBMCs demonstrated to be a potential biomarker of effect to assess the EDs impact on reproductive health.