This paper describes a novel application of angular spectrum diffraction theory as prior knowledge for unsupervised deep learning processes that avoids the conventional requirement for a massive learning terahertz image dataset. This approach to unsupervised deep learning succeeds in the extraction of amplitude and phase information, but it may sacrifice some generalization ability. When combined with supervised deep learning techniques that utilize a curated image dataset, improvements in generalization ability with reduced noise and improved edge sharpness and contrast are realized. The combined deep learning approach outperforms traditional algorithms, particularly for relatively simple scenes.