ATH/PDMS systems are often used as potting compounds in electronic assemblies to guard the electronics from shock, vibration, corrosive agents and moisture. In this study, we use dynamic rheology and confocal/optical microscopy to understand the dramatic effects miniscule levels of water have on the microstructure and corresponding rheological behavior of PDMS filled with ATH. In the absence of water, PDMS containing 20 wt.% ATH readily flows, exhibiting viscoelastic behavior with some weak particle flocculation. However, the addition of only 0.045 wt.% water to the system results in the formation of a sample-spanning, self-supporting physical gel that exhibits an elastic modulus (G') five orders of magnitude higher than the water-free system. A structure formation mechanism consisting of hydration layer formation followed by inter-particle water bridging has been proposed to explain the observed behavior. Recovery of the original viscoelastic fluid is demonstrated by adding molecular sieves or zeolites to the fully flocculated system. The recovery can likely be attributed to the adsorption of water by the sieves and the corresponding breakup of water bridges between the ATH particles. Based on the proposed mechanism, a variety of other polar and non-polar solvents have been found to induce physical gelation in ATH/PDMS dispersions with gel modulus being related to the Hildebrand solubility parameter mismatch between the solvent and PDMS fluid.