Abstract Background Dipeptidyl peptidase 4 (DPP4) inhibitors improve glycemic control by promoting GLP1-mediated glucose-dependent insulin secretion and suppression of glucagon. Sitagliptin and vildagliptin have been shown to improve insulin sensitivity in patients with type 2 diabetes mellitus (T2DM). However, these patients had uncontrolled blood glucose at inclusion; therefore, the improvement in insulin sensitivity observed in these studies could be attributed to the drug per se and/or reduction in glucotoxicity. This study examines the effect of linagliptin on insulin sensitivity and β-cell function in patients with well-controlled T2DM. Methods Thirty patients with T2DM of duration ≤5 years, and having HbA1c < 7.5% were randomized to receive linagliptin, voglibose or placebo (n = 10 each), and were followed up for 6 months. Insulin sensitivity was assessed by hyperinsulinemic-euglycemic clamp, and insulin secretory response was measured by basal (M0) and postprandial (M1) β-cell function, and area under curve (AUC) for C-peptide during mixed meal tolerance test. Results The median HbA1c of the study subjects at inclusion was 6.9% and there was no significant difference among the groups in terms of age, duration of diabetes, body mass index (BMI), HbA1c, insulin sensitivity, AUC of C-peptide and M0 and M1 at baseline. At the end of the study, there was a modest reduction in HbA1c (− 0.2%) in the linagliptin group, and a significant decrease (− 0.8%) in the voglibose group, as compared to placebo (p = 0.038). However, there were no significant differences in insulin sensitivity, M0 and M1 and AUC of C-peptide, within, or among the groups. Conclusion Linagliptin modestly improves glycemic profile in patients with well controlled T2DM; however, it may not have an effect on insulin sensitivity in these patients. Trial registration Retrospectively Registered in Clinicaltrials.gov (ID number, NCT02097342). Registered: March 27, 2014.