Large protein machines are tightly regulated through allosteric communication channels. Here we demonstrate the involvement of ultrafast conformational dynamics in allosteric regulation of ClpB, a hexameric AAA+ machine that rescues aggregated proteins. Each subunit of ClpB contains a unique coiled-coil structure, the middle domain (M domain), proposed as a control element that binds the co-chaperone DnaK. Using single-molecule FRET spectroscopy, we probe the M domain during the chaperone cycle and find it to jump on the microsecond time scale between two states, whose structures are determined. The M-domain jumps are much faster than the overall activity of ClpB, making it an effectively continuous, tunable switch. Indeed, a series of allosteric interactions are found to modulate the dynamics, including binding of nucleotides, DnaK and protein substrates. This mode of dynamic control enables fast cellular adaptation and may be a general mechanism for the regulation of cellular machineries.