Mercury (Hg) causes severe nephrotoxicity in subjects with excess exposure. This work attempted to identify whether a natural medicine—rhubarb—has protective effects against mercuric chloride (HgCl2)-induced acute renal failure (ARF), and which of its components contributed most to the treatment. Total rhubarb extract (TR) were separated to the total anthraquinones (TA), the total tannins (TT) and remaining component extract (RC). Each extract was orally pre-administered to rats for five successive days followed by HgCl2 injection to induce kidney injury. Subsequently, renal histopathology and biochemical examinations were performed in vitro to evaluate the protective effects. Pharmacological studies showed that TR and TA, but not TT or RC manifested significant protection activity against HgCl2-induced ARF. There were also significant declines of serum creatine, urea nitrogen values and increases of total protein albumin levels in TR and TA treated groups compared to HgCl2 alone (p < 0.05). At last, the major components in TA extract were further identified as anthraquinones by liquid chromatography coupled mass spectroscopy. This study thus provides observational evidences that rhubarb could ameliorate HgCl2-induced ARF and its anthraquinones in particular are the effective components responsible for this activity in rhubarb extract.