Accumulating neuroimaging studies in humans have shown that acupuncture can modulate a widely distributed brain network in mild cognitive impairment (MCI) and Alzheimer’s disease (AD) patients. Acupuncture at different acupoints could exert different modulatory effects on the brain network. However, whether acupuncture at real or sham acupoints can produce different effects on the brain network in MCI or AD patients remains unclear. Using resting-state fMRI, we reported that acupuncture at Taixi (KI3) induced amplitude of low-frequency fluctuation (ALFF) change of different brain regions in MCI patients from those shown in the healthy controls. In MCI patients, acupuncture at KI3 increased or decreased ALFF in the different regions from those activated by acupuncture in the healthy controls. Acupuncture at the sham acupoint in MCI patients activated the different brain regions from those in healthy controls. Therefore, we concluded that acupuncture displays more significant effect on neuronal activities of the above brain regions in MCI patients than that in healthy controls. Acupuncture at KI3 exhibits different effects on the neuronal activities of the brain regions from acupuncture at sham acupoint, although the difference is only shown at several regions due to the close distance between the above points.