The Retrospective Assessment of the Lithium Response Phenotype Scale (Alda scale) is the most widely used clinical measure of lithium response phenotypes. We assess its performance against recommended psychometric and clinimetric standards.We used data from the Consortium for Lithium Genetics and a French study of lithium response phenotypes (combined sample >2500) to assess reproducibility, responsiveness, validity and interpretability of the A scale (assessing change in illness activity), the B scale and its items (assessing confounders of response) and the previously established response categories derived from the Total Score for the Alda scale.The key findings are that the B scale is vulnerable to error measurement. For example, some items contribute little to overall performance of the Alda scale (e.g. B2) and that the B scale does not reliably assess a single construct (uncertainty in response). Machine learning models indicate that it may be more useful to employ and algorithm for combining the ratings of individual B items in a sequence that clarifies the noise to signal ratio instead of using a composite score.This study highlights three important topics. First, empirical approaches can help determine which aspects of the performance of any scale can be improved. Second, the B scale of the Alda is best applied as a multidimensional index (identifying several independent confounders of the assessment of response). Third, an integrated science approach to precision psychiatry is vital, otherwise phenotypic misclassifications will undermine the reliability and validity of findings from genetics and biomarker studies.