Eddy current pulsed thermography (ECPT), a new emerging nondestructive testing and evaluation (NDT&E) technique, has been applied for a wide range of conductive materials. The acquired image sequences contain valuable information in both spatial and time domain. ECPT techniques mainly use a specific frame to detect and quantify the defects. However, selection of specific frame from transient thermal image video to maximize the contrast of thermal variation and defect pattern from complex geometrical samples remain a challenge. In order to accurately find anomalous patterns from the transient thermal pattern for defect detection and further quantitative NDE, this paper employs a single channel blind source separation algorithm. This method enables spatial and time patterns to be extracted according to the whole transient response behavior without any training knowledge. In this paper, both mathematical and physical models are discussed, and the basis of the proper selection of contrast image is given. In addition, the artificial slot and thermal fatigue natural crack are applied to validate the proposed method.