Olive fruits, as well as their corresponding oil, represent an interesting source of phytochemicals, mainly phenolic compounds, which arise as secondary metabolites, resulting from the plant’s response to biotic and abiotic stresses. Therefore, olive fruits from three distinct cultivars (“Cobrançosa,” “Galega Vulgar,” and “Picual”) grown in Portugal and displaying different degree of resistance to biotic and abiotic stresses have been studied in relation to the concentration of total phenolic compounds, orthodiphenols and flavonoids, besides antioxidant capacity (DPPH, ABTS, FRAP, and ORACFL), in three maturation stages and two distinct harvest seasons (2012-2013). Generally, a decrease of all phenolic contents throughout the maturation stages has been observed, while, concerning cultivar, green olives of “Cobrançosa” showed the highest values for all contents assessed, denoting a strong influence of the genetic background. The same trend has not been observed regarding antioxidant activity, since Cobrançosa and Galega Vulgar cultivars presented the highest values only for the DPPH and ORACFL assays. Moreover, multivariate analyses pointed to the preponderance of the cultivars’ phenolic composition in the semiripe stage for the resistance to biotic stress, with “Galega” the most susceptible cultivar, presenting the lowest contents at this maturation stage, whereas “Picual” displayed the most pronounced phytochemical response.