As mammals develop, they encounter increasing social complexity in the surrounding world. In order to survive, mammals must show appropriate behaviors toward their mates, offspring, and same-sex conspecifics. Although the behavioral effects of the neuropeptide arginine vasopressin (AVP) have been studied in a variety of social contexts, the effects of this neuropeptide on multimodal sensory processing has received less attention. AVP is widely distributed through sensory regions of the brain and has been demonstrated to modulate olfactory, auditory, gustatory, and visual processing. Here we review the evidence linking AVP to the processing of social stimuli in sensory regions of the brain and explore how sensory processing can shape behavioral responses to these stimuli. In addition, we address the interplay between hormonal and neural AVP in regulating sensory processing of social cues. Because AVP pathways show plasticity during development, early life experiences may shape life-long processing of sensory information. Furthermore, disorders of social behavior such as autism and schizophrenia that have been linked with AVP also have been linked with dysfunctions in sensory processing. Together, these studies suggest that AVP’s diversity of effects on social behavior across a variety of mammalian species may result from the effects of this neuropeptide on sensory processing.