Gastrointestinal cancers account for 37% of all cancer deaths worldwide, underlining the need to further investigate modifiable factors for gastrointestinal cancer risk and prognosis. This review summarizes the corresponding evidence for physical activity (PA), including, briefly, possible biological mechanisms. Despite high public health relevance, there is still a scarcity of studies, especially for tertiary prevention. Besides the convincing evidence of beneficial effects of PA on colon cancer risk, clear risk reduction for gastroesophageal cancer was identified, as well as weak indications for pancreatic cancer. Inverse associations were observed for liver cancer, yet based on few studies. Only for rectal cancer, PA appeared to be not associated with cancer risk. With regard to cancer-specific mortality of the general population, published data were rare but indicated suggestive evidence of protective effects for colon and liver cancer, and to a lesser extent for rectal and gastroesophageal cancer. Studies in cancer patients on cancer-specific and total mortality were published for colorectal cancer only, providing good evidence of inverse associations with post-diagnosis PA. Overall, evidence of associations of PA with gastrointestinal cancer risk and progression is promising but still limited. However, the already available knowledge further underlines the importance of PA to combat cancer.