An Ag-graphene layers-coated H-shaped photonic crystal fiber (PCF) surface plasmon resonance (SPR) sensor with a U-shaped grooves open structure for refractive index (RI) sensing is proposed and numerically simulated by the finite element method (FEM). The designed sensor could solve the problems of air-holes material coating and analyte filling in PCF. Two big air-holes in the x-axis produce a birefringence phenomenon leading to the confinement loss and sensitivity of x-polarized light being much stronger than y-polarized. Graphene is deposited on the layer of silver in the grooves; its high surface to volume ratio and rich π conjugation make it a suitable dielectric layer for sensing. The effect of structure parameters such as air-holes size, U-shaped grooves depth, thickness of the silver layer and number of graphene layers on the sensing performance of the proposed sensor are numerical simulated. A large analyte RI range from 1.33 to 1.41 is calculated and the highest wavelength sensitivity is 12,600 nm/RIU. In the linear RI sensing region of 1.33 to 1.36; the average wavelength sensitivity we obtained can reach 2770 nm/RIU with a resolution of 3.61 × 10 RIU. This work provides a reference for developing a high-sensitivity; multi-parameter measurement sensor potentially useful for water pollution monitoring and biosensing in the future.