Probabilistic methods such as mutual information and Bayesian networks have become a major category of tools for the reconstruction of regulatory relationships from quantitative biological data. In this chapter, we describe the theoretic framework and the implementation for learning gene regulatory networks using high-order mutual information via the MI3 method (Luo et al. (2008) BMC Bioinformatics 9, 467; Luo (2008) Gene regulatory network reconstruction and pathway inference from high throughput gene expression data. PhD thesis). We also cover the closely related Bayesian network method in detail.