Cause-of-death data is fundamental for understanding population health trends and inequalities as well as designing and evaluating public health interventions. A significant proportion of global deaths, particularly in low- and middle-income countries (LMICs), do not have medically certified causes assigned. In such settings, verbal autopsy (VA) is a widely adopted approach to estimate disease burdens by interviewing caregivers of the deceased. Recently, latent class models have been developed to model the joint distribution of symptoms and perform probabilistic cause-of-death assignment. A large number of latent classes are usually needed in order to characterize the complex dependence among symptoms, making the estimated symptom profiles challenging to summarize and interpret. In this paper, we propose a flexible Bayesian tensor decomposition framework that balances the predictive accuracy of the cause-of-death assignment task and the interpretability of the latent structures. The key to our approach is to partition symptoms into groups and model the joint distributions of group-level symptom sub-profiles. The proposed methods achieve better predictive accuracy than existing VA methods and provide a more parsimonious representation of the symptom distributions. We show our methods provide new insights into the clustering patterns of both symptoms and causes using the PHMRC gold-standard VA dataset.