One of the main objectives of watermarking is to achieve a better tradeoff between robustness and high visual quality of a host image. In recent years, there has been a significant development in gray-level image watermarking using fractal-based method. This paper presents a human visual system (HVS) based fractal watermarking method for color images. In the proposed method, a color pixel is considered as a 3-D vector in RGB space. And a general form of 3 × 3 matrix is utilized as the scaling operator. Meanwhile, the luminance offset vector is substituted by the range block mean vector. Then an orthogonalization fractal color coding method is achieved to obtain very high image quality. We also show that the orthogonalization fractal color decoding is a mean vector-invariant iteration. So, the range block mean vector is a good place for hiding watermark. Furthermore, for consistency with the characteristics of the HVS, we carry out the embedding process in the CIE La*b* space and incorporate a just noticeable difference (JND) profile to ensure the watermark invisibility. Experimental results show that the proposed method has good robustness against various typical attacks, at the same time, with an imperceptible change in image quality.