Images are often corrupted by impulse noise. In this paper, an alternating direction minimization with continuation algorithm is modified and iteratively used to remove the impulse noise in images by exploring its self-similarity. A patch-based nonlocal operator and sparse representation are married in the l1-l1 optimization model to be solved. Simulation results demonstrate that the proposed algorithm outperforms typical denoising methods in terms of preserving edges and textures for both salt-and-pepper noise and random-valued impulse noise. It can be also applied to suppress impulse noise-like artifacts in real mural images.