Young children with multiple disabilities (e.g., both cognitive and motor disabilities) are confronted with severe limitations in language development from birth and later on. Stimulating the adult-child communication can decrease these limitations. Within LinguaBytes, a three-year research program, we try to stimulate language development by developing an interactive and adaptive play and learning environment, incorporating tangible objects and multimedia content, based on interactive storytelling and anchored instruction. The development of a product for such a heterogeneous user group presents substantial challenges. We use a Research-through-Design method, that is, an iterative process of developing subsequent experiential prototypes and then testing them in real-life settings, for example, a center for rehabilitation medicine. This article gives an outline of the development of the LinguaBytes play and learning environment from the earliest studies up to the current prototype, CLICK-IT.