Structural equation models are used to model the relationships between latent constructs and observable behaviors such as survey responses. Researchers are often interested in testing nested models to determine whether additional constraints that create a more parsimonious model are also supported by the data. A popular statistical tool for nested model comparison is the chi-square difference test. However, there is some evidence that this test performs suboptimally when the unrestricted model is misspecified. In this paper, we examine the type I error rate of the difference test within the context of single-group confirmatory factor analyses when the less restricted model is misspecified but the constraints imposed by the restricted model are correct. Using empirical simulations and analytic approximations, we find that the chi-square difference test is robust to many but not all forms of realistically sized misspecification in the unrestricted model.