Inspired by Smith, Ben-Moshe suggests that we should only accommodate conscientious objections (COs) in medicine based on moral beliefs that are true, or which closely approximate to the truth. He suggests that we can identify moral truths by consulting our consciences when our consciences adopt the standpoint of an impartial spectator. He also suggests some (surprisingly modest) changes to our current practices in regard to the management of CO in medicine that would be needed were his proposal to be adopted. Here, I argue that both Smith and Ben-Moshe underestimate the difficulties involved in adopting the standpoint of an impartial spectator. In particular, both authors fail to recognise the extent to which cognitive bias and ideological commitments prevent many of us from identifying the standpoint of an impartial spectator and also prevent us from realising that we are failing to be impartial. I also consider some different changes to current practices that would be needed if we were to take on Ben-Moshe's approach to CO in medicine while also recognising the difficulties involved in adopting the standpoint of an impartial spectator.