Excessive cadmium (Cd) accumulation in grains of rice (Oryza sativa L.) is a risk to food security. The transporters in the nodes of rice are involved in the distribution of mineral elements including toxic elements to different tissues such as grains. However, the mechanism of Cd accumulation in grains is largely unknown. Here, we report a node-expressed transporter gene, OsCCX2, a putative cation/calcium (Ca) exchanger, mediating Cd accumulation in the grains of rice. Knockout of OsCCX2 caused a remarkable reduction of Cd content in the grains. Further study showed that disruption of this gene led to a reduced root-to-shoot translocation ratio of Cd. Moreover, Cd distribution was also disturbed in different levels of internode and leaf. OsCCX2 is localized to plasma membrane, and OsCCX2 is mainly expressed in xylem region of vascular tissues at the nodes. OsCCX2 might function as an efflux transporter, responsible for Cd loading into xylem vessels. Therefore, our finding revealed a novel Cd transporter involved in grain Cd accumulation, possibly via a Ca transport pathway in the nodes of rice.