To achieve effective intracellular anticancer drug delivery, the polymeric vesicles supplemented with the pH-responsive outlayered gels as a delivery system of doxorubicin (DOX) were developed from self-assembly of the lipid/polypeptide adduct, distearin grafted poly(γ-glutamic acid) (poly(γ-GA)), followed by sequential deposition of chitosan and poly(γ-GA-co-γ-glutamyl oxysuccinimide)-g-monomethoxy poly(ethylene glycol) in combination with in situ covalent cross-linking on assembly surfaces. The resultant gel-caged polymeric vesicles (GCPVs) showed superior performance in regulating drug release in response to the external pH change. Under typical physiological conditions (pH 7.4 and 37 °C) at which the γ-GA/DOX ionic pairings remained mostly undisturbed, the dense outlayered gels of GCPVs significantly reduced the premature leakage of the uncomplexed payload. With the environmental pH being reduced from pH 7.4 to 4.7, the drug liberation was appreciably promoted by the massive disruption of the ionic γ-GA/DOX complexes along with the significant swelling of nanogel layers upon the increased protonation of chitosan chain segments. After being internalized by HeLa cells via endocytosis, GCPVs exhibited cytotoxic effect comparable to free DOX achieved by rapidly releasing the payload in intracellular acidic endosomes and lysosomes. This strongly implies the great promise of such unique GCPVs as an intracellular drug delivery carrier for potential anticancer treatment.