Two new terpene glycosides (1–2) along with two known analogs (3–4) were obtained from the root of Sanguisorba officinalis, which is a common traditional Chinese medicine (TCM). Their structures were elucidated by nuclear magnetic resonance (NMR), electrospray ionization high resolution mass spectrometry (HRESIMS), and a hydrolysis reaction, as well as comparison of these data with the literature data. Compounds 1–4 exhibited anti-inflammatory properties in vitro by attenuating the production of inflammatory mediators, such as nitric oxide (NO) as well as tumor necrosis factor-α (TNF-α) and interleukin-6 (IL-6). An anti-inflammatory assay based on the zebrafish experimental platform indicated that compound 1 had good anti-inflammatory activity in vivo by not only regulating the distribution, but also by reducing the amount of the macrophages of the zebrafish exposed to copper sulfate.