In the past decade, Tanzania has seen a rapid rise in the number of people who inject drugs (PWID), specifically heroin. While the overall HIV prevalence in Tanzania has declined recently to 5.6%, in 2009, the HIV prevalence among PWID remains alarmingly high at 35%. In this paper, we describe how the Tanzania AIDS Prevention Program (TAPP), Médecins du Monde France (MdM-F), and other organisations have been at the forefront of addressing this public health issue in Africa, implementing a wide array of harm reduction interventions including medication-assisted treatment (MAT), needle and syringe programs (NSP), and “sober houses” for residential treatment in the capital, Dar es Salaam, and in Zanzibar. Looking toward the future, we discuss the need to (1) extend existing services and programs to reach more PWID and others at risk for HIV, (2) develop additional programs to strengthen existing programs, and (3) expand activities to include structural interventions to address vulnerabilities that increase HIV risk for all Tanzanians.