To evaluate the translation of a paper high-risk checklist for PICU patients at risk of clinical deterioration to an automated clinical decision support tool.Retrospective, observational cohort study of an automated clinical decision support tool, the PICU Warning Tool, adapted from a paper checklist to predict clinical deterioration events in PICU patients within 24 hours.Two quaternary care medical-surgical PICUs-The Children's Hospital of Philadelphia and Cincinnati Children's Hospital Medical Center.The study included all patients admitted from July 1, 2014, to June 30, 2015, the year prior to the initiation of any focused situational awareness work at either institution.We replicated the predictions of the real-time PICU Warning Tool by retrospectively querying the institutional data warehouse to identify all patients that would have flagged as high-risk by the PICU Warning Tool for their index deterioration.The primary exposure of interest was determination of high-risk status during PICU admission via the PICU Warning Tool. The primary outcome of interest was clinical deterioration event within 24 hours of a positive screen. The date and time of the deterioration event was used as the index time point. We evaluated the sensitivity, specificity, positive predictive value, and negative predictive value of the performance of the PICU Warning Tool. There were 6,233 patients evaluated with 233 clinical deterioration events experienced by 154 individual patients. The positive predictive value of the PICU Warning Tool was 7.1% with a number needed to screen of 14 patients for each index clinical deterioration event. The most predictive of the individual criteria were elevated lactic acidosis, high mean airway pressure, and profound acidosis.Performance of a clinical decision support translation of a paper-based tool showed inferior test characteristics. Improved feasibility of identification of high-risk patients using automated tools must be balanced with performance.