A new nonlinear optical (NLO) material, RbNa5Ca5(CO3)8, has been synthesized by the hydrothermal method. The crystal structure is established by single-crystal X-ray diffraction. RbNa5Ca5(CO3)8 crystallizes in the hexagonal crystal system with space group P63mc (No. 186). The structure of RbNa5Ca5(CO3)8 can be described as the adjacent infinite [CaCO3]∞ layers lying in the a-b plane bridged through standing-on-edge [CO3] groups by sharing O atoms (two-fold coordinated) to build a framework with four types of tunnels running through the b-axis. The Rb, Na, and [Na0.67Ca0.33] atoms reside in these tunnels, respectively. The measurement of second harmonic generation (SHG) indicated that RbNa5Ca5(CO3)8 is a phase-matchable material, which had SHG responses of approximately 1×KH2PO4 (KDP). Meanwhile, the results from the UV-VIS diffuse reflectance spectroscopy study of the powder samples indicated that the UV cut-off edges of RbNa5Ca5(CO3)8 is about 203 nm.