Recently the method of specifying multiple regions of interest (ROI) based image retrieval has been suggested. However it measures the similarity of the images without proper consideration of the spatial layouts of the ROIs and thus fails to accurately reflect the intent of the user. In this paper, we propose a new similarity measurement using the relative layouts of the ROIs. The proposed method divides images into blocks of certain size and extracted MPEG-7 dominant colors from the blocks overlapping with the user-designated ROIs to measure their similarities with the target images. At this point, similarity was weighted when the relative location of the ROIs in the query image and the target image was the same. The relative location was calculated by four directions (i.e. up, down, left and right) of the basis ROI. The proposed method by an experiment using MPEG-7 XM shows that its performance is higher than the global image retrieval method or the retrieval method that does not consider the relative location of ROIs.