The Arp2/3 complex generates branched actin networks that exert pushing forces onto different cellular membranes. WASH complexes activate Arp2/3 complexes at the surface of endosomes and thereby fission transport intermediates containing endocytosed receptors, such as α5β1 integrins. How WASH complexes are assembled in the cell is unknown. Here, we identify the small coiled-coil protein HSBP1 as a factor that specifically promotes the assembly of a ternary complex composed of CCDC53, WASH, and FAM21 by dissociating the CCDC53 homotrimeric precursor. HSBP1 operates at the centrosome, which concentrates the building blocks. HSBP1 depletion in human cancer cell lines and in amoebae phenocopies WASH depletion, suggesting a critical role of the ternary WASH complex for WASH functions. HSBP1 is required for the development of focal adhesions and of cell polarity. These defects impair the migration and invasion of tumor cells. Overexpression of HSBP1 in breast tumors is associated with increased levels of WASH complexes and with poor prognosis for patients.