Thanks to the wide range of benefits it provides to teaching and learning, portfolio assessment has maintained widespread popularity in language education over the last few decades. However, the practical use of this assessment method is still subject to debates, particularly about the lack of clear definitions and empirical validations of the constructs underlying the assessment. This problem can be addressed by research into portfolio scale development and examination and this article reports on the process of investigating the psychometric properties of a scale assessing the portfolio-based writing competence of Vietnamese students who speak English as a foreign language (EFL). The psychometric validation in this investigation involved applying different Rasch models, including multidimensional, partial credit and many-facet models, to examine the characteristics of the scale items. The findings support the use of the scale with mostly good item functioning and acceptable raters' consistency in using the scale. Finally, only one item addressing the length of writing is removed from the developed scale and the items assessing the planning stage of writing in writing portfolios are flagged for further inspection in a larger scale study. Implications for using the scale to improve the quality of teaching and assessment of writing via portfolios can be drawn.