In this study, we investigated the physicochemical properties of soil, and the diversity and structure of the soil ammonia-oxidizing archaea (AOA) community, when subjected to fertilizer treatments for over 35 years. We collected soil samples from a black soil fertilization trial in northeast China. Four treatments were tested: no fertilization (CK); manure (M); nitrogen (N), phosphorus (P), and potassium (K) chemical fertilizer (NPK); and N, P, and K plus M (MNPK). We employed 454 high-throughput pyrosequencing to measure the response of the soil AOA community to the long-term fertilization. The fertilization treatments had different impacts on the shifts in the soil properties and AOA community. The utilization of manure alleviated soil acidification and enhanced the soybean yield. The soil AOA abundance was increased greatly by inorganic and organic fertilizers. In addition, the community Chao1 and ACE were highest in the MNPK treatment. In terms of the AOA community composition, Thaumarchaeota and Crenarchaeota were the main AOA phyla in all samples. Compared with CK and M, the abundances of Thaumarchaeota were remarkably lower in the MNPK and NPK treatments. There were distinct shifts in the compositions of the AOA operational taxonomic units (OTUs) under different fertilization management practices. OTU51 was the dominant OTU in all treatments, except for NPK. OTU79 and OTU11 were relatively abundant OTUs in NPK. Only Nitrososphaera AOA were tracked from the black soil. Redundancy analysis indicated that the soil pH and soil available P were the two main factors that affected the AOA community structure. The abundances of AOA were positively correlated with the total N and available P concentrations, and negatively correlated with the soil pH.