Theories on embodied cognition emphasize that our mind develops by processing and inferring structures given the encountered bodily experiences. Here we propose a distributed neural network architecture that learns a stochastic generative model from experiencing bodily actions. Our modular system learns from various manifolds of action perceptions in the form of (i) relative positional motion of the individual body parts, (ii) angular motion of joints, as well as (iii) relatively stable top-down action identities. By Hebbian learning, this information is spatially segmented in separate neural modules that provide embodied state codes as well as temporal predictions of the state progression inside and across the modules. The network is generative in space and time, thus, being able to predict both, missing sensory information as well as next sensory information. We link the developing encodings to visuo-motor and multimodal representations that appear to be involved in action observation. Our results show that the system learns to infer action types as well as motor codes from partial sensory information by emulating observed actions with the own developing body model. We further evaluate the generative capabilities by showing that the system is able to generate internal imaginations of the learned types of actions without sensory stimulation, including visual images of the actions. The model highlights the important roles of motor cognition and embodied simulation for bootstrapping action understanding capabilities. We conclude that stochastic generative models appear very suitable for both, generating goal-directed actions, as well as predicting observed visuo-motor trajectories and action goals.