To develop novel therapeutic agents with anticancer activities, two series of novel 7-amino-[1,2,4]triazolo[4,3-f]pteridinone, and 7-aminotetrazolo[1,5-f]pteridinone derivatives were designed and synthesized. All compounds were tested for anti-proliferative activities against five cancer cell lines. The structure-activity relationships (SARs) studies were conducted through the variation in two regions, the moiety of A ring and the terminal aniline B on pteridinone core. 1-Methyl-1,2,4-triazole derivative L with 2,6-dimethylpiperazine showed the most potent antiproliferative activity against A549, PC-3, HCT116, MCF-7 and MDA-MB-231 cell lines with IC values of 0.16 μM, 0.30 μM, 0.51 μM, 0.30 μM, and 0.70 μM, respectively. Combined with the results of the molecular docking and enzymatic studies, the PLK1 was very likely to be one of the drug targets of compound L. Furthermore, to clarify the anticancer mechanism of compound L, further explorations in the bioactivity were conducted. The results showed that compound L obviously inhibited proliferation of A549 cell lines, induced a great decrease in mitochondrial membrane potential leading to apoptosis of cancer cells, suppressed the migration of tumor cells, and arrested G1 phase of A549 cells.