Hyperspectral anomaly detection plays an important role in the field of remote sensing. It provides a way to distinguish interested targets from the background without any prior knowledge. The majority of pixels in the hyperspectral dataset belong to the background, and they can be well represented by several endmembers, so the background has a low-rank property. Anomalous targets usually account for a tiny part of the dataset, and they are considered to have a sparse property. Recently, the low-rank and sparse matrix decomposition (LRaSMD) technique has drawn great attention as a method for solving anomaly detection problems. In this letter, a new anomaly detection method based on LRaSMD and cluster weighting is proposed. We concentrate on the sparse part, which contains most of anomaly information, and calculate the initial anomaly matrix based on this part. To suppress background regions and discriminate anomalies from the background more distinctly, a weighting strategy in terms of the clustering result is used, and then the anomaly matrix is updated. The judgement of anomalies is made according to the responses on the matrix. Our proposed method considers the characteristics of anomalies from the spectral dimension and the spatial distribution simultaneously. Experiments on three hyperspectral datasets demonstrate the outstanding performance of the proposed method.