Categorical knowledge and episodic memory have traditionally been viewed as separate lines of inquiry. Here, we present a perspective on the interrelatedness of categorical knowledge and reconstruction from memory. We address three underlying questions: What knowledge do people bring to the task of remembering? How do people integrate that knowledge with episodic memory? Is this the optimal way for the memory system to work? In the review of five studies spanning four category domains (discrete, continuous, temporal and linguistic), we evaluate the relative contribution and the structure of influence of categorical knowledge on long-term episodic memory. These studies suggest a robustness of peoples’ knowledge of statistical regularities in the environment, and provide converging evidence of the quality and influence of category knowledge on reconstructive memory. Lastly, we argue that combining categorical knowledge and episodic memory is an efficient strategy of the memory system.