The extensive uptake of telehealth has considerably transformed health care delivery since the beginning of the COVID-19 pandemic and has imposed tremendous challenges to its large-scale implementation and adaptation. Given the shift in paradigm from telehealth as an alternative mechanism of care delivery to telehealth as an integral part of the health system, it is imperative to take a systematic approach to identifying barriers to, opportunities for, and the overall impact of telehealth implementation amidst the current pandemic. In this work, we apply a human factors framework, the Systems Engineering Initiative for Patient Safety model, to guide our holistic analysis and discussion of telehealth implementation, encompassing the health care work system, care processes, and outcomes.