Titanium alloys have several advantages, such as a high strength-to-weight ratio. However, the machinability of titanium alloys is not as good as its mechanical properties. Many machining processes have been used to fabricate titanium alloys. Among these machining processes, electrical discharge machining (EDM) has the advantage of processing efficiency. EDM is based on thermoelectric energy between a workpiece and an electrode. A pulse discharge occurs in a small gap between the workpiece and electrode. Then, the material from the workpiece is removed through melting and vaporization. However, defects such as cracks and notches are often detected at the boundary of holes fabricated using EDM and the irregular profile of EDM holes reduces product quality. In this study, an innovative method was proposed to estimate the effect of EDM parameters on the surface quality of the holes. The method combining the finite element method and image processing can rapidly evaluate the stress concentration factor of a workpiece. The stress concentration factor was assumed as an index of EDM process performance for estimating the surface quality of EDM holes. In EDM manufacturing processes, Ti-6Al-4V was used as an experimental material and, as process parameters, pulse current and pulse on-time were taken into account. The results showed that finite element simulations can effectively analyze stress concentration in EDM holes. Using high energy during EDM leads to poor hole quality, and the stress concentration factor of a workpiece is correlated to hole quality. The maximum stress concentration factor for an EDM hole was more than four times that for the same diameter of the undamaged hole.