Infrared NDT techniques are suitable for detecting near-surface and sub-surface defects in composite materials. However, significant detection challenges remain while data volume is quantitatively limited for training weak contrast between defective and non-defective regions. In this paper, we propose an end-to-end memory linked knowledge domain with transfer few-shot learning segmentation network with a wider general model that solves the problems of insufficient data support. The proposed method is shown to attain high detection performance for multiple types of specimens by using cross domain adaption. By improving the support branch, a foreground-background joint guidance linked with memory bank is proposed. This provides better guidance information for the query branch. In addition, the intra-class and inter-class loss metric distance between defective and non-defective features is increased for better segmentation performance. In order to enable interpretability of the model, robust experiments have been conducted to detect the inner debond on multiple carbon fiber reinforced polymer (CFRP) composites. A comparative analysis is presented and compared with the state-of-the-art machine learning algorithms.