Resistive random-access memory plays a key role in non-volatile and neuromorphic artificial electronic devices. In this work, we fabricated Ta/TaOx/AlN/Pt resistive memory devices with the inserted AlN layer to improve the performance. The devices have better stability, lower threshold voltage (≈0.37 V), longer retention time (>104 s), and faster switching speed (9 ns) than those without the AlN film layer. More importantly, when different pulse parameters were applied, two phenomena, abrupt jumps in conductance and gradual change in conductance, were obtained. Furthermore, the biological synaptic functions were simulated, including the spiking-time-dependent plasticity and the paired-pulse facilitation. The Ta/TaOx/AlN/Pt resistive memory devices offer promising features; hence, they are good candidates for next-generation electronic devices for chip systems.