In recent years, privacy awareness is concerned due to many Internet services have chosen to use encrypted agreements. In order to improve the quality of service (QoS), the network encrypted traffic behaviors are classified based on machine learning discussed in this paper. However, the traditional traffic classification methods, such as IP/ASN (Autonomous System Number) analysis, Port-based and deep packet inspection, etc., can classify traffic behavior, but cannot effectively handle encrypted traffic. Thus, this paper proposed a hybrid traffic classification (HTC) method based on machine learning and combined with IP/ASN analysis with deep packet inspection. Moreover, the majority voting method was also used to quickly classify different QoS traffic accurately. Experimental results show that the proposed HTC method can effectively classify different encrypted traffic. The classification accuracy can be further improved by 10% with majority voting as K = 13. Especially when the networking data are using the same protocol, the proposed HTC can effectively classify the traffic data with different behaviors with the differentiated services code point (DSCP) mark.