The evolutionarily conserved target of rapamycin complex 1 (TORC1) couples an array of intra- and extracellular stimuli to cell growth, proliferation and metabolism, and its deregulation is associated with various human pathologies such as immunodeficiency, epilepsy, and cancer. Among the diverse stimuli impinging on TORC1, amino acids represent essential input signals, but how they control TORC1 has long remained a mystery. The recent discovery of the Rag GTPases, which assemble as heterodimeric complexes on vacuolar/lysosomal membranes, as central elements of an amino acid signaling network upstream of TORC1 in yeast, flies, and mammalian cells represented a breakthrough in this field. Here, we review the architecture of the Rag GTPase signaling network with a special focus on structural aspects of the Rag GTPases and their regulators in yeast and highlight both the evolutionary conservation and divergence of the mechanisms that control Rag GTPases.