The structural architecture and the anatomical connectivity of the human brain show different organizational principles at distinct spatial scales. Histological staining and light microscopy techniques have been widely used in classical neuroanatomical studies to unravel brain organization. Using such techniques is a laborious task performed on 2-dimensional histological sections by skilled anatomists possibly aided by semi-automated algorithms. With the recent advent of modern magnetic resonance imaging (MRI) contrast mechanisms, cortical layers and columns can now be reliably identified and their structural properties quantified post mortem. These developments are allowing the investigation of neuroanatomical features of the brain at a spatial resolution that could be interfaced with that of histology. Diffusion MRI and tractography techniques, in particular, have been used to probe the architecture of both white and gray matter in three dimensions. Combined with mathematical network analysis, these techniques are increasingly influential in the investigation of the macro-, meso- and microscopic organization of brain connectivity and anatomy, both in vivo and ex vivo. Diffusion MRI-based techniques in combination with histology approaches can therefore support the endeavor of creating multimodal atlases that take into account the different spatial scales or levels on which the brain is organized. The aim of this review is to illustrate and discuss the structural architecture and the anatomical connectivity of the human brain at different spatial scales and how recently developed diffusion MRI techniques can help investigate these.