Induction thermography has been applied as an emerging non-destructive testing and evaluation technique for a wide range of conductive materials. The infrared vision sensing acquired image sequences contain valuable information in both spatial and time domain. However, automatic and precisely extracting defect pattern from thermal video remains a challenge. In order to accurately find anomalous patterns for defect detection and further quantitative nondestructive evaluation, we propose an automatic relevance determination approach with adaptive variational Bayes for sub-group sparse decomposition. A subset of scale parameters is driven to a small low bound in the inference, with the pruning the corresponding spurious components. In addition, an internal sub-sparse grouping as well as adaptive fine-tuned is built into the proposed algorithm to control the sparsity. Experimental tests on both artificially and nature defects and comparisons with other methods have been conducted to verify the efficacy of the proposed method.