The CD8+ memory T cell population is heterogeneous, and it is unclear which subset(s) optimally mediate the central goal of the immune system—protection against infection. Here we investigate the protective capacities of CD8+ T cell subsets present at the memory stage of the immune response. We show that a population of CD8+ T cells bearing markers associated with effector cells (KLRG1hi, CD27lo, T-bethi, Eomeslo) persisted to the memory phase and provided optimal control of Listeria monocytogenes and vaccinia virus, despite weak recall proliferative responses. After antigen-specific boosting, this population formed the predominant secondary memory subset and maintained superior pathogen control. The effector-like memory subset displayed a distinct pattern of tissue distribution and localization within the spleen, and their enhanced capacity to eliminate Listeria involved specialized utilization of cytolysis. Together, these data suggest that long-lived effector CD8+ T cells are optimal for protective immunity against certain pathogens.