In this paper, the problem of fast low-energy halo-to-halo transfers between Sun–planet systems is discussed under ephemeris constraints. According to the structure of an invariant manifold, employing an invariant manifold and planetary gravity assist to save fuel consumption is analyzed from the view of orbital energy. Then, a pseudo-manifold is introduced to replace the invariant manifold in such a way that more transfer opportunities are allowed. Fast escape and capture can be achieved along the pseudo-manifold. Furthermore, a global searching method that is based on patched-models is proposed to find an appropriate transfer trajectory. In this searching method, the trajectory is divided into several segments that can be designed under simple dynamical models, and an analytical algorithm is developed for connecting the segments. Earth–Mars and Earth–Venus halo-to-halo transfers are designed to demonstrate the proposed approach. Numerical results show that the transfers that combine the pseudo-manifolds and planetary gravity assist can offer significant fuel consumption and flight time savings over traditional transfer schemes.