Epidemiological data indicate that more than 50 % of patients with newly-diagnosed rectal cancer are older than 70 years, with rising numbers expected over the next decades. Treatment decision-making is challenging in elderly and frail patients with rectal cancer, whereas standardized treatment guidelines for this patient cohort are lacking. Elderly and frail rectal cancer patients are often considered by surgeons as unfit to undergo radical surgery as the risk of surgical complications and postoperative mortality rises with increasing age and comorbidity. Furthermore, these patients often receive no treatment at all, resulting in local and/or systemic disease progression with associated symptoms and impaired quality of life (QoL). Recent data from randomized trials in young fit patients with early stage rectal cancer indicate that RT dose escalation can be safely delivered using external beam (chemo)radiotherapy (EBRT) followed by endoluminal radiotherapeutic modalities, such as contact X-ray brachytherapy (CXB) or high-dose rate endorectal brachytherapy (HDR-BT). However, prospective studies testing this therapeutic concept in elderly and frail patients remain limited. Here, we review the current evidence in the epidemiology and the management of elderly and frail patients with rectal cancer. We summarize the potential of RT dose escalation to achieve long-term local control of the primary tumour, prevent disease-related morbidity, improve QoL and even organ preservation. Future perspectives and open questions will be discussed as well.