Metal nanocrystals exhibit important optoelectronic and photocatalytic functionalities in response to light. These dynamic energy conversion processes have been commonly studied by transient optical probes to date, but an understanding of the atomistic response following photoexcitation has remained elusive. Here, we use femtosecond resolution electron diffraction to investigate transient lattice responses in optically excited colloidal gold nanocrystals, revealing the effects of nanocrystal size and surface ligands on the electron-phonon coupling and thermal relaxation dynamics. First, we uncover a strong size effect on the electron - phonon coupling, which arises from reduced dielectric screening at the nanocrystal surfaces and prevails independent of the optical excitation mechanism (e.g., inter- and intra-band). Second, we find that surface ligands act as a tuning parameter for hot carrier cooling. Particularly, gold nanocrystals with thiol-based ligands show significantly slower carrier cooling as compared to amine-based ligands under intra-band optical excitation due to electronic coupling at the nanocrystal / ligand interfaces. Finally, we spatio-temporally resolve thermal transport and heat dissipation in photoexcited nanocrystal films by combining electron diffraction with stroboscopic elastic scattering microscopy. Taken together, we resolve the distinct thermal relaxation timescales ranging from 1 ps to 100 ns associated with the multiple interfaces through which heat flows at the nanoscale. Our findings provide insights into optimization of gold nanocrystals and their thin films for photocatalysis and thermoelectrics applications.