Hydrazine borane N2H4BH3 and alkali derivatives (i.e., lithium, sodium and potassium hydrazinidoboranes MN2H3BH3 with M = Li, Na and K) have been considered as potential chemical hydrogen storage materials. They belong to the family of boron- and nitrogen-based materials and the present article aims at providing a timely review while focusing on fundamentals so that their effective potential in the field could be appreciated. It stands out that, on the one hand, hydrazine borane, in aqueous solution, would be suitable for full dehydrogenation in hydrolytic conditions; the most attractive feature is the possibility to dehydrogenate, in addition to the BH3 group, the N2H4 moiety in the presence of an active and selective metal-based catalyst but for which further improvements are still necessary. However, the thermolytic dehydrogenation of hydrazine borane should be avoided because of the evolution of significant amounts of hydrazine and the formation of a shock-sensitive solid residue upon heating at >300 °C. On the other hand, the alkali hydrazinidoboranes, obtained by reaction of hydrazine borane with alkali hydrides, would be more suitable to thermolytic dehydrogenation, with improved properties in comparison to the parent borane. All of these aspects are surveyed herein and put into perspective.