Recent evidence suggests a rather gradual developmental trajectory for processing vertical relational face information, lasting well into late adolescence (de Heering and Schlitz, 2012). Results from another recent study (Tanaka et al., 2014) indicate that children and young adolescents use a smaller spatial integration field for faces than do adults, which particularly affects assessment of long-range vertical relations. Here we studied sensitivity to replacement of eyes and eyebrows (F), variation of inter-eye distance (H) and eye height (V) in young adolescents (11-12 years), young (21-25 years) and middle-age adults (51-62 years). In order to provide a baseline for potential age effects the sensitivity to all three types of face manipulations was calibrated to equal levels for the young adults group. Both the young adolescents and the middle age adults showed substantially lower sensitivity compared to young adults, but only the young adolescents had selective impairment for V relational changes. Their inversion effects were at similar levels for all types of face manipulations, while in both adult groups the inversion effects for V were considerably stronger than for H or F changes. These results suggest that young adolescents use a limited spatial integration field for faces, and have not reached a mature state in processing vertical configural cues. The H-V asymmetry of inversion effects found for both adult groups indicates that adults integrate across the whole face when they view upright stimuli. However, the notably lower sensitivity of middle-age adults for all types of face manipulations, which was accompanied by a strong general same bias, suggests early age-related decline in attending cues for facial difference.