Microorganisms play important roles in soil improvement. Therefore, clarifying the contribution of environmental factors in shaping the microbial community structure is beneficial to improve soil fertility in karst rocky desertification areas. Here, the bacterial community structures of eight rhizospheric soil samples collected from perennial fruit plantations were analysed using an Illumina HiSeq2500 platform. The diversity and abundance of bacteria in rocky desertification areas were significantly lower than those in non-rocky desertification areas, while the bacterial community structure was not significantly different between root surface and non-root surface soils in the same rhizospheric soil samples. Proteobacteria predominated in rocky desertification areas, while Actinobacteria predominated in non-rocky desertification areas. Correlation analysis revealed that water-soluble phosphorus content (r = 0.8258), latitude (r = 0.7556), altitude (r = 0.7501), and the age of fruit trees (r = 0.7321) were positively correlated with the bacterial community structure, while longitude, pH, and total phosphorus content did not significantly influence the soil bacterial community structure. As water-soluble phosphorus content is derived from insoluble phosphorus minerals, supplementing phosphorus-solubilising bacteria to soils in rocky desertification areas is a feasible strategy for accelerating the dissolution of insoluble phosphorus minerals and improving agricultural production and environment ecology.