This study explored the factor structure of psychiatric nurses’ job-related stress and examined the specificity of the related stressors using the job stressor scale of the Brief Job Stress Questionnaire (BJSQ). The stressor scale of the BJSQ was administered to 296 nurses and assistant nurses. Answers were examined statistically. Exploratory factor analysis was performed to identify factor structures; two factors (overload and job environment) were valid. Confirmatory factor analysis was conducted to examine the two-factor structure and found 11 items with factor loadings of >0.40 (model 1), 13 items with factor loadings from 0.30 to <0.40 (model 2), and 17 items with factor loadings from 0.20 to <0.30 (model 3) for one factor; model 1 demonstrated the highest goodness of fit. Then, we observed that the two-factor structure (model 1) showed a higher goodness of fit than the original six-factor structure. This differed from subscales based on general workers’ job-related stressors, suggesting that the factor structure of psychiatric nurses’ job-related stressors is specific. Further steps may be necessary to reduce job-related stress specifically related to overload including attention to many needs of patients and job environment including complex ethical dilemmas in psychiatric nursing.