This paper describes an English as a Foreign Language (EFL) speaking and listening test designed and piloted through the mobile application Telegram. The test was designed to diagnose the speaking and listening skills of first-year students at the Indonesian university UNIROW Tuban before they arrive on campus. The mobile delivery of the test, which can be accessed by smart phone and computer, was aimed to evaluate if learning apps can increase communicative learner interaction with authentic materials in English. The mobile test takes advantage of the vast amount of multimedia that can be transferred online, such as video and podcasts. Test-takers respond to Speaking Section prompts by recording themselves and answer Listening Section questions by clicking on the correct multiple choice options. All test-taker responses are sent to the administrator of the test, created through an inline Telegram bot. After piloting the test with students in a first-year conversation class at UNIROW Tuban, the test underwent a series of analyses, including item facility, item discrimination, split-half reliability, inter-rater reliability, and subtest relationships. These analyses are important for verifying the test’s validity, reliability, and practicality. Overall, the test seems to be an important tool not only for diagnosing students listening and speaking needs, but also to increase interest in learning and practicing conversational English. Although the test was designed for one university in particular, it (along with variations) can be used in similar contexts throughout the world.