Previous studies on the effects of emotional context on memory for centrally presented neutral items have obtained inconsistent results. And in most of those studies subjects were asked to either make a connection between the item and the context at study or retrieve both the item and the context. When no response for the contexts is required, how emotional contexts influence memory for neutral items is still unclear. Thus, the present study attempted to investigate the influences of four types of emotional picture contexts on recognition memory of neutral words using both behavioral and event-related potential (ERP) measurements. During study, words were superimposed centrally onto emotional contexts, and subjects were asked to just remember the words. During test, both studied and new words were presented without the emotional contexts and subjects had to make old/new judgments for those words. The results revealed that, compared with the neutral context, the negative contexts and positive high-arousing context impaired recognition of words. ERP results at encoding demonstrated that, compared with items presented in the neutral context, items in the positive and negative high-arousing contexts elicited more positive ERPs, which probably reflects an automatic process of attention capturing of high-arousing context as well as a conscious and effortful process of overcoming the interference of high-arousing context. During retrieval, significant FN400 old/new effects occurred in conditions of the negative low-arousing, positive, and neutral contexts but not in the negative high-arousing condition. Significant LPC old/new effects occurred in all conditions of context. However, the LPC old/new effect in the negative high-arousing condition was smaller than that in the positive high-arousing and low-arousing conditions. These results suggest that emotional context might influence both the familiarity and recollection processes.