Sound sensing finds wide applications in various fields, such as underwater detection, structural health monitoring, and medical diagnosis, to name just a few. Based on our previously developed MEMS-on-fiber sensors, showing the advantages of low cost, small volume, and high performance, a three-dimensional ultrasonic localization system employing four such sensors was established in this work. A time difference of arrival (TDOA) algorithm was utilized to analyze the acquired data and then calculate the accurate position of the ultrasonic signal source. Plenty of practical measurements were performed, and the derived localization deviation in the region of 2 m × 2 m × 1 m was about 2-5 mm. Outside this region, the deviation tended to increase due to the directional sensitivity existing in these sensors. As a result, for a more accurate localization requirement, more sensing probes are needed in order to depict a completely suitable application situation for MEMS technology.