Compare experts' ability to differentiate malignant and benign causes of facial nerve paralysis (FNP) using the initial presenting magnetic resonance image (MRI) for each patient.This retrospective case-controlled study compared MRIs for 9 patients with a malignant cause for FNP, 8 patients with Bell's palsy, and 9 cochlear implant patients serving as controls. The initial presenting MRI for each condition was used such that raters were evaluating real-world rather than optimal studies. Three blinded expert raters independently evaluated each segment of the facial nerve for abnormalities, provided a diagnosis, and graded MRI quality. Cohen's and Light's kappa were used to calculate interrater reliability and overall index of agreement, respectively.MRI protocols for the malignancy group were universally suboptimal. There was poor agreement among raters for abnormalities of the facial nerve along the brainstem (0.13), geniculate (0.10), tympanic segment (0.12), and mastoid segment (0.13); moderate agreement along the cisternal segment (0.58) and internal auditory canal (0.55); and fair agreement along the labyrinthine segment (0.26) and extratemporal segment (0.36). Agreement regarding final diagnosis was fair (0.37) when compared to the true diagnosis. There were 2 false negative interpretations (failure to correctly identify malignancy) and 1 false positive interpretation.MRI for FNP is often initially performed with an incorrect protocol and thus may fail to reliably differentiate neoplastic from inflammatory FNP even when interpreted by experienced clinicians. Nevertheless, expert readers correctly diagnosed 87.5% of malignant causes of FNP despite these limitations.