Chronic inflammation is a major cause of human cancers. The environmental factors, such as microbiome, dietary components, and obesity, provoke chronic inflammation in the prostate, which promotes cancer development and progression. Crosstalk between immune cells and cancer cells enhances the secretion of intercellular signaling molecules, such as cytokines and chemokines, thereby orchestrating the generation of inflammatory microenvironment. Tumor-associated macrophages (TAMs) and myeloid-derived suppressor cells (MDSCs) play pivotal roles in inflammation-associated cancer by inhibiting effective anti-tumor immunity. Anti-inflammatory agents, such as aspirin, metformin, and statins, have potential application in chemoprevention of prostate cancer. Furthermore, pro-inflammatory immunity-targeted therapies may provide novel strategies to treat patients with cancer. Thus, anti-inflammatory agents are expected to suppress the “vicious cycle” created by immune and cancer cells and inhibit cancer progression. This review has explored the immune cells that facilitate prostate cancer development and progression, with particular focus on the application of anti-inflammatory agents for both chemoprevention and therapeutic approach in prostate cancer.