Temperature-responsive flexibility in metal-organic frameworks (MOFs) appeals to the imagination. The ability to transform upon thermal stimuli while retaining a given crystalline topology is desired for specialized sensors and actuators. However, rational design of such shape-memory nanopores is hampered by a lack of knowledge on the nanoscopic interactions governing the observed behavior. Using the prototypical MIL-53(Al) as a starting point, we show that the phase transformation between a narrow-pore and large-pore phase is determined by a delicate balance between dispersion stabilization at low temperatures and entropic effects at higher ones. We present an accurate theoretical framework that allows designing breathing thermo-responsive MOFs, based on many-electron data for the dispersion interactions and density-functional theory entropy contributions. Within an isoreticular series of materials, MIL-53(Al), MIL-53(Al)-FA, DUT-4, DUT-5 and MIL-53(Ga), only MIL-53(Al) and MIL-53(Ga) are proven to switch phases within a realistic temperature range.