Intimate partner sexual violence (IPSV) among emerging adults, including heterosexuals and sexual minorities in the Chinese population, is poorly understood. Focusing on college students, the objectives of the present study were to estimate the prevalence of IPSV among Chinese emerging adults, examine the association between sexual orientation and IPSV, and evaluate the mental health and quality of life of survivors of IPSV. Data were collected from four university campuses in Hong Kong. A total of 1,015 participants were included in the data analysis. The prevalence of IPSV as identified by Conflict Tactics Scale (CTS)-2 was 12.1%. There was no gender difference in the prevalence of IPSV. Multiple logistic regression found that being older; having experienced childhood sexual abuse and having a smoking habit; and belonging to a sexual minority were factors associated with IPSV. Multiple linear regression found that IPSV survivors were more likely to have higher levels of anxiety and depression, more severe psychosomatic symptoms, and poorer quality of life in three domains: psychological, social relationships, and environment when compared with those without IPSV experience. The documented factors underscored the importance of awareness that men and sexual minorities might incur IPSV, which should receive more attention in IPSV prevention programs. Also, interventions for IPSV survivors should be targeted at improving their mental health and quality of life and sex education should place more emphasis on sexual consent and sexual health in dating relationships.