Understanding the mechanisms that regulate the differentiation and maintenance of CD8+ memory T cells is fundamental to the development of effective T cell based vaccines. Memory cell differentiation is influenced by the cytokines that accompany T cell priming, the history of previous antigen encounters, and the tissue sites into which memory cells migrate. These cues combine to influence the developing CD8+ memory pool, and recent work has revealed the importance of multiple transcription factors, metabolic molecules, and surface receptors in revealing the type of memory cell that is generated. Paired with increasingly meticulous subsetting and sorting of memory populations, we now know the CD8+ memory pool to be phenotypically and functionally heterogeneous in nature. This includes both recirculating and tissue resident memory populations, and cells with varying degrees of inherent longevity and protective function. These data point to the importance of tailored vaccine design. Here we discuss how the diversity of the memory CD8+ T cell pool challenges the notion that ‘one size fits all’ for pathogen control, and how distinct memory subsets may be suited for distinct aspects of protective immunity.