Abstract This study addresses the problem of appropriately assessing the spoken language ability of non-native graduate students functioning as international teaching assistants (ITAs) in English-speaking environments in general and that of a Canadian university in particular. It examines the problem with reference to the needs of ITAs in actual contexts of language use in the light of two validity standards of 'authenticity' and 'directness' (Messick, 1989) and the model of language testing proposed by Bachman and Palmer (1996). The paper summarizes the results of a needs assessment carried out among three major groups of participants at the University of Victoria: administrators and graduate advisors, undergraduate students and the ITAs themselves. Test constructs are then formulated based on the results of the needs analysis. It is also shown how test constructs are translated into the communicative task types that would involve ITAs in performances from which inferences can be made with respect to their language abilities. Finally, the resulting assessment device and its rating instrument together with an account of the pilot administration of the test are introduced. Conclusions have been drawn with respect to the reliability and practicality of the test.