In recent decades, gaming technology has been accepted as a feasible method for complementing traditional clinical practice, especially in neurorehabilitation; however, the viability of using 3D Virtual Reality (VR) for the assessment of upper limb motor function has not been fully explored. For that purpose, we developed a VR-based version of the Box and Blocks Test (BBT), a clinical test for the assessment of manual dexterity, as an automated alternative to the classical procedure. Our VR-based BBT (VR-BBT) integrates the traditional BBT mechanics into gameplay using the Leap Motion Controller (LMC) to capture the user’s hand motion and the Oculus Rift headset to provide a fully immersive experience. This paper focuses on evaluating the validity of our VR-BBT to reliably measure the manual dexterity in a sample of patients with Parkinson’s Disease (PD). For this study, a group of twenty individuals in a mild to moderate stage of PD were recruited. Participants were asked to perform the physical BBT (once) and our proposed VR-BBT (twice) system, separately. Correlation analysis of collected data was carried out. Statistical analysis proved that the performance data collected by the VR-BBT significantly correlated with the conventional assessment of the BBT. The VR-BBT scores have shown a significant association with PD severity measured by the Hoehn and Yahr scale. This fact suggests that the VR-BBT could be used as a reliable indicator for health improvements in patients with PD. Finally, the VR-BBT system presented high usability and acceptability rated by clinicians and patients.