The development of mouse embryos can be partially recapitulated by combining embryonic stem cells (ESCs), trophoblast stem cells (TS), and extra-embryonic endoderm (XEN) stem cells to generate embryo-like structures called ETX embryos. Although ETX embryos transcriptionally capture the mouse gastrula, their ability to recapitulate complex morphogenic events such as gastrulation is limited, possibly due to the limited potential of XEN cells. To address this, we generated ESCs transiently expressing transcription factor Gata4, which drives the extra-embryonic endoderm fate, and combined them with ESCs and TS cells to generate induced ETX embryos (iETX embryos). We show that iETX embryos establish a robust anterior signaling center that migrates unilaterally to break embryo symmetry. Furthermore, iETX embryos gastrulate generating embryonic and extra-embryonic mesoderm and definitive endoderm. Our findings reveal that replacement of XEN cells with ESCs transiently expressing Gata4 endows iETX embryos with greater developmental potential, thus enabling the study of the establishment of anterior-posterior patterning and gastrulation in an in vitro system.