Breaks filled with different break activities often interrupt cognitive performance in everyday life. Previous studies have reported that both enhancing and deteriorating effects on challenging ongoing tasks such as working memory updating, depend on the type of break activity. However, neural mechanisms of these break-related alterations in working memory performance have not been studied, to date. Therefore, we conducted a brain imaging study to identify the neurobiological correlates of effects on the n-back working memory task related to different break activities. Before performing the n-back task in the magnetic resonance imaging (MRI) scanner, young adults were exposed to break activities in the MRI scanner involving (i) eyes-open resting, (ii) listening to music, and (iii) playing the video game "Angry Birds". Heart rate was measured by a pulse oximeter during the experiment. We found that increased heart rate during gaming as well as decreased relaxation levels after a video gaming break was related to poorer n-back task performance, as compared to listening to music. On the neural level, video gaming reduced supplementary motor area activation during working memory performance. These results may indicate that video gaming during a break may affect working memory performance by interfering with arousal state and frontal cognitive control functions.