Oolong tea is a semi-fermented tea that can be classified into different grades based on flavor, aroma, and other factors. In this paper, we used a combination of metabolomics (HPLC and SPME) and microbiomics (16S rRNA) to explore the substances and microbial causes affecting the quality of oolong tea. The results showed that six taste substances such as epicatechin, and soluble sugar were significantly differentiated taste substances, 22 flavor substances such as ethylhexanol, pentenal were significantly differentiated aroma substances, and 109 microorganisms such as Paenibacillus and Haemophilus were significantly differentiated microorganisms. The content of aroma substances was more closely associated with oolong tea quality, and cis-2-pentenol, 3,5-octadien-2-one, and 2,5-dimethylpyrazine had high correlations with oolong tea tasting scores. Microorganisms such as Prevotella, Schaalia, and Niallia were positively associated with oolong tea quality. This study established the association between oolong tea quality and substances and microorganisms, which provides a feasible direction for improving oolong tea grade. By refining the processing techniques of oolong tea, such as the fine manipulation of green tea leaves, and enhancing specific microorganisms and metabolites, this study provides a reference for improving the quality of oolong tea. It also offers potential research directions for upgrading the quality of low-grade oolong tea through deep processing industries.