Einstein gravity in both 3 and 4 dimensions, as well as some interesting generalizations, can be written as gauge theories in which the connection is a Cartan connection for geometry modeled on a symmetric space. The relevant models in 3 dimensions include Einstein gravity in Chern-Simons form, as well as a new formulation of topologically massive gravity, with arbitrary cosmological constant, as a single constrained Chern-Simons action. In 4 dimensions the main model of interest is MacDowell-Mansouri gravity, generalized to include the Immirzi parameter in a natural way. I formulate these theories in Cartan geometric language, emphasizing also the role played by the symmetric space structure of the model. I also explain how, from the perspective of these Cartan-geometric formulations, both the topological mass in 3d and the Immirzi parameter in 4d are the result of non-simplicity of the Lorentz Lie algebra so(3,1) and its relatives. Finally, I suggest how the language of Cartan geometry provides a guiding principle for elegantly reformulating any 'gauge theory of geometry'.