To rate one’s individual olfactory performance is difficult and in many cases differs clearly from validated objective olfactory performance measures. This study aimed to investigate the basis for this measurement drift between objective and subjective olfactory performance evaluation. In absence of an actual odor, one may imagine an olfactory stimulus to evaluate his subjective olfactory performance. Therefore, the impact of the vividness of mental images on self-evaluation of smell performance in patients with mild to severe olfactory dysfunction and healthy controls was investigated. Fifty-nine patients with peripheral olfactory dysfunction ranging from reduced olfactory function (hyposmia) to complete loss of olfactory perception (anosmia) and 16 healthy controls were included. Olfactory performance was assessed using the Sniffin’ Sticks battery, the vividness of olfactory mental images was evaluated using the Vividness of Olfactory Imagery Questionnaire (VOIQ).Decreased vividness of odor images was obtained for anosmic patients, and a trend of poorer odor imagery was determined in hyposmic patients. Multiple regression analyses revealed the VOIQ score as significant predictor for olfactory self-evaluation for hyposmic patients and healthy controls. In contrast, for anosmic patients, the only significant predictor for self-rating of olfactory performance was the TDI score. The results of this study indicate that sensory perception and mental images are closely related to each other. Furthermore, subjects who were able to perceive odors, even to a smaller extent, rely on the vividness of their mental odor images to evaluate their olfactory performance. In contrast, anosmic patients rather trust in their knowledge that they are not able to perceive odors. We are therefore able to subjectively rate our olfactory performance levels, if we are not able to perceive odors, but not if we are able to perceive olfactory input.