This paper proposes an Externally Recurrent Neural Network (ERNN) for approximating the unknown dynamics of complex nonlinear systems and time series prediction. The proposed model utilizes the present as well as delayed values of the system outputs as well as of the external input. The weight update equations are tested for their boundedness by applying the Lyapunov stability method. Further, the error convergence proof is also given. The proposed model is put to test by considering various nonlinear examples and its performance is also compared with other state of the art methods. The results obtained in the present study indicate that the method is efficient and has provided accurate results.