Electricity load forecasting is becoming one of the key issues to solve energy crisis problem, and time-series Bayesian Neural Network is one popular method used in load forecast models. However, it has long running time and relatively strong dependence on time and weather factors at a residential level. To solve these problems, this article presents an improved Bayesian Neural Networks (IBNN) forecast model by augmenting historical load data as inputs based on simple feedforward structure. From the load time delays correlations and impact factors analysis, containing different inputs, number of hidden neurons, historic period of data, forecasting time range, and range requirement of sample data, some advices are given on how to better choose these factors. To validate the performance of improved Bayesian Neural Networks model, several residential sample datasets of one whole year from Ausgrid have been selected to build the improved Bayesian Neural Networks model. The results compared with the time-series load forecast model show that the improved Bayesian Neural Networks model can significantly reduce calculating time by more than 30 times and even when the time or meteorological factors are missing, it can still predict the load with a high accuracy. Compared with other widely used prediction methods, the IBNN also performs a better accuracy and relatively shorter computing time. This improved Bayesian Neural Networks forecasting method can be applied in residential energy management.