In the paper a referral system to assist the medical experts in the screening/referral of diabetic retinopathy is suggested. The system has been developed by a sequential use of different existing mathematical techniques. These techniques involve speeded up robust features (SURF), K-means clustering and visual dictionaries (VD). Three databases are mixed to test the working of the system when the sources are dissimilar. When experiments were performed an area under the curve (AUC) of 0.9343 was attained. The results acquired from the system are promising.