Abstract Poor user experience is caused by the current unstable operation of the two-wheel self-balancing vehicle control system, the high failure rate, and the poor accuracy and sensitivity of the control system (prone to deviation). Therefore, the work studied the two-wheel self-balancing vehicle system with the visual recognition of vehicle attitude. Kalman filtering and PID control algorithm were adopted to reduce the high frequency interference of the accelerometer and the low frequency error of the gyroscope, improve the response speed and control precision of motor to error, and ensure vertical control. The road images were acquired by a charge-coupled device vision sensor with the edge information extracted by the Laplacian operator, which ensures autonomous navigation control of a balance car. Based on the calculation of the distance between road axis and the existence detection of forward obstacles, the early warning mechanism was established through the calculation of the group agent to improve the safety performance of a balance vehicle. Experiments showed the improved control system has good stability, fast walking speed, strong anti-interference, and high security.