Virtual reality (VR) through a head-mounted display (HMD) can provide new experiences. However, it remains unclear how the characteristics of HMDs affect users' memory. To use HMDs more effectively and appropriately in several applied fields, including education, it is necessary to clarify what characteristics of HMDs affect users' memory. A head-tracking function mounted on an HMD helps to detect the user's head direction to enable a simulation experience akin to the real world. When we experience a simulation on an HMD, we actively perceive the visual world. In this study, we assessed how active/passive viewing affects users' memory of VR content. We conducted a psychological experiment in which participants watched a movie on an HMD. In the active viewing condition, the presented view changed depending on the participant's head direction. In the passive viewing condition, the presented view was a recorded movie that was shown to the participants in the active viewing condition. All participants took a memory test about the content presented in the movie on the day of viewing and 2 weeks later. The results showed that performance on the memory test in the active viewing condition was significantly lower than that in the passive viewing condition after 2 weeks. This result indicated that active viewing in VR inhibited users' memory compared to passive viewing. The current study contributes to the development of new VR techniques, such as educational learning.