In this publication a method for tracking a speaker with acoustical information by means of a microphone array is presented. A sound source localization algorithm based on the time delays of arrival of sound waves in microphone pairs provides initial position estimates. These significantly varying estimates are spatially filtered by an adaptive Kalman filter to obtain a smoothed trajectory of the speaker’s movement. Experimental results with real data are presented for a variety of scenarios recorded in a noisy and reverberant office room environment.