We propose an adaptive neuro-fuzzy inference system (ANFIS) with an incremental tree structure based on a context-based fuzzy C-means (CFCM) clustering process. ANFIS is a combination of a neural network with the ability to learn, adapt and compute, and a fuzzy machine with the ability to think and to reason. It has the advantages of both models. General ANFIS rule generation methods include a method employing a grid division using a membership function and a clustering method. In this study, a rule is created using CFCM clustering that considers the pattern of the output space. In addition, multiple ANFISs were designed in an incremental tree structure without using a single ANFIS. To evaluate the performance of ANFIS in an incremental tree structure based on the CFCM clustering method, a computer performance prediction experiment was conducted using a building heating-and-cooling dataset. The prediction experiment verified that the proposed CFCM-clustering-based ANFIS shows better prediction efficiency than the current grid-based and clustering-based ANFISs in the form of an incremental tree.