Introduction
Electromagnetic inverse-scattering problems (ISPs) aim to reconstruct the physical properties of unknown objects using the scattered field. Due to the non-destructive property, the imaging techniques based on ISPs have a wide range of important applications, such as non-destructive evaluation, microwave imaging, remote sensing, biomedical imaging, and civil measurement [Reference Ireland, Bialkowski and Abbosh1–Reference Monte, Erricolo, Soldovieri and Wicks5]. However, ISPs are challenging to address because they are often intrinsically ill-posed and nonlinear.
A large variety of inverse-scattering algorithms have been developed to retrieve the unknown objects more efficiently and reliably. Relevant solutions for ISPs can be classified into linear and nonlinear methods. In the category of linear method, the first-order Born approximation (BA) [Reference Poli, Oliveri and Massa6] is used to describe the total field, and thus linearize the relationship between objects contrast and received scattered field. The main regularization methods for linear ill-posed ISPs include truncated singular value decomposition [Reference Shea, Van Veen and Hagness7] and low-rank constraints [Reference Zhou, Mo, Wang and Duan8]. By and large, linear methods can reconstruct the permittivity of non-strong scatterers with high quality and require lower-computing resources. However, for strong scatterers, the performance of linear approaches would dramatically decrease. In the kind of nonlinear method, in order to cope with strong scatterers in a better fashion, the multiple scattering effect should be considered in physical modeling, and nonlinear iterative optimization approaches have been developed, such as Born iterative method [Reference Wang and Chew9], contrast source inversion method [Reference Van Den Berg and Kleinman10, Reference Song, Yu and Liu11], subspace optimization method (SOM) [Reference Chen12, Reference Xu, Zhong and Wang13], etc. These iterative approaches minimize the objective function, which quantifies the mismatch between the calculated data and the measured one to reconstruct unknown scatterers' permittivity distribution. By contrast, nonlinear iterative inverse methods can reconstruct spatial distribution of strong scatterers. Nevertheless, it is often sensitive to initialization values and converges at a relatively slow speed, so that not suitable for real-time reconstruction. Moreover, even if the iterative methods are adopted, it is relatively difficult to reconstruct the complex scatterers. This is because the data equations used to solve nonlinear inverse problems are usually underdetermined [Reference Bertero and Boccacci14]. The reconstructed distribution still can be distorted and can deviate away from the ground truth even when the error between the calculated scattered field and the measured field is small.
In recent years, deep learning has achieved promising performance in a variety of engineering applications [Reference Sun, Xia and Kamilov15–Reference Massa, Marcantonio, Chen, Li and Salucci17]. Many deep neural networks (DNNs) have been proposed to provide solutions for ISPs [Reference Caorsi and Gamba18, Reference Rekanos19]. For example, Li et al. [Reference Li, Wang, Teixeira, Liu, Nehorai and Cui20] proposed a DNN-based nonlinear ISP inversion method by exploiting the DNN architecture combined with the iterative method of non-linear inverse scattering. Chen et al. [Reference Wei and Chen29] employed the result of the traditional back-propagation method as the input of DNN to obtain a better estimation of the contrast. Wei et al. [Reference Wei and Chen21] exploited an induced current learning method. It contains several strategies to integrate the domain knowledge of physical iterative methods into the neural network structure. The above-mentioned deep learning-based inverse-scattering research has mainly focused on deep learning-assisted optimization and nonlinear iterative physical model-driven methods. Zhou et al. [Reference Zhou, Ouyang, Li, Liu and Liu22] presented a linear model-based network (LMN) with a learnable regularizer for solving linear ISPs. Although LMN is almost comparable to advance nonlinear iterative methods from the perspective of the relative error of reconstructed permittivity, the texture details of the scatterers will be lost to a certain extent when the experimental environment further degrades.
A convolutional neural network (CNN) is the most widely used network for inverse-scattering imaging on which many state-of-the-art approaches rely. However, while achieving relatively accurate imaging, solutions of CNN optimization problems often lack high-frequency content which results in perceptually unsatisfying reconstruction of images. In generative adversarial networks (GANs), Goodfellow et al. [Reference Goodfellow, Pouget-Abadie, Mirza, Xu, Warde-Farley, Ozair and Bengio23] put forward a new framework for estimating generative models through adversarial processes. Besides, the GAN may offer a way to break current limits in supervised learning [Reference Chen, Wei, Li and Rocca24]. In this paper, an adversarial framework is proposed to strengthen the capabilities of the BA model to predict complex profiles. We choose BA, because BA can adapt to low-frequency diffraction tomography and therefore is more suitable for a wider frequency range [Reference Chew25]. More specifically, we inherit the spirit of SRGAN [Reference Ledig, Theis, Huszár, Caballero, Cunningham, Acosta and Shi26] and try to apply the generative adversarial model to approximate the contrast mapping instead of the popularly-used CNN. According to the processing mode of linear ISPs, the roughly reconstructed result serves as the input of the designed generative network, which outputs the fine reconstructed image of the relative permittivity. However, the reconstructed complex contour will usually lack high-frequency details when only the generative network is employed. Therefore, the discriminative network is introduced to ensure the fidelity of the reconstruction results. The purpose of the discriminative network differentiates the reconstructed scatterers and the ground truths. In addition, the gradient updated by the generator network is not directly derived from the data sample, but uses back-propagation from a discriminator network. Training generative network and discriminative network alternately in a confrontational manner until reaching the Nash equilibrium to make the reconstructed spatial distribution is more similar to the ground truth in semantics and style.
The remainder of this paper is organized as follows. In Section “Problem formulation for ISPs,” the general model and basic theory of the ISPs are introduced. In Section “Generative network architecture,” the structure of the generative network is presented. In Section “Discriminative network and loss function,” the structure of the discriminative network and loss function are introduced. In Section “Numerical results,” numerical examples demonstrate the accuracy and effectiveness of the LM-GAN. And conclusions are drawn in Section “Conclusion.”
Notion: $\bar{\bar{\chi }}$
denotes the matrix. Then, A H denotes the conjugate transpose of a matrix A. || ⋅ ||F denotes the Frobenius norm of a matrix. Finally, in the reconstructed images, the units of the horizontal and vertical coordinates are meters.
Problem formulation for ISPs
Figure 1 presents the representative schematic diagram of ISPs. To define the ISPs conveniently, the case of two-dimensional transverse magnetic is considered, where the longitude direction is along $\hat{z}$
. One or more nonmagnetic scatterers are located in a domain of interest (DoI) in free-space background. The sources and receivers are equally placed outside the DoI, and their relevant position vectors are recorded as r j, j = 1, 2, …, N i and r q, q = 1, 2, …, N s, respectively. A total number of N i line sources radiate harmonic electromagnetic waves through the unknown scatterers. Then, the scattered electric field generated by each incidence is measured by an array of N s antennas, so the size of the obtained total data of the scattered field is N iN s. The unknown scatterers are located in the DoI where the background medium is evenly distributed, so the relevant permittivity of the background is ɛ0 and the permeability is μ 0.
Two-dimensional configuration of electromagnetic ISPs.
The total field in the DoI can be described by using the Lippmann–Schwinger equation:
where E tot(r) and E inc(r) denote the total and incident electric fields, respectively. $k_0 = \omega \sqrt {\mu _0\varepsilon _0}$
represents the background medium wave number. ω is the angular frequency of the incident electromagnetic wave. G(r, r ′) is the two-dimensional free-space Green's function. The diagonal matrix stands for the contrast of reconstructed object scatterer whose diagonal element is χ(r n,n) = ɛ(r n) − 1, ɛ(r n) is the relative permittivity at r n. The scattered field satisfies the integral equation:
where E sca(r) is the scattered field on the measurement surface S.
To streamline the problem, the above two integral equations are discretized, thus yield two discretized formulations. The total field in the DoI can be expressed as:
The scattered field satisfies the discretized equation:
From formulations (3) and (4), it is concluded that the relative permittivity of the unknown objects can be determined from the measured scattering field. In this paper, only linear ISPs are considered. The integral equation can be solved for $\bar{\bar{\chi }}$
under the BA:
We also add a learnable regularization function $Z_\vartheta ( \bar{\bar{\chi }})$
concerning the prior of $\bar{\bar{\chi }}$
in equation (7) to mitigate the ill-posedness of linear ISPs:
Generative network architecture
By setting $Z_\vartheta ( \bar{\bar{\chi }})$
to be network-contained, it yields
The objective function (7) is decomposed into two sub-problems equations (9) and (10):
where $\bar{\bar{B}}_n$
is a learnable regularizer CNN estimator that depends on the network parameters $\vartheta$
. In addition, $\bar{\bar{B}}_n$
is applied to stabilize the imaging process.
By calculating the gradient of sub-problem equation (9) and letting it to be zero, it attains:
The schematic diagram of the generative network framework is depicted in Fig. 2. The generative network is initialized with ${\bar{\chi }}_0 = \bar{A}^H\bar{E}^{sca}$
, where $\bar{\bar{A}}^H = ( \bar{\bar{G}}_S\,{\rm diag}( \bar{\bar{E}}^{tot}) ) ^H$
. Afterward, $\bar{\bar{B}}_{n + 1}$
and $\bar{\bar{\chi }}_{n + 1}$
are updated alternately by the U-net [Reference Ronneberger, Fischer and Brox27] based denoiser step (10) and conjugate gradient step (11).
Overall network architecture of LM-GAN. The regularization sub-network employed a U-net structure, which consists of four layers of up-sampling and four layers of down-sampling. The discriminative network contains five layers of convolution.
To sum it up, the above update rules can be considered as a generative network consisting of K-stages (K = 1, 2, …, N). Each stage consists of the learnable regularization CNN module and the data-consistency (DC) unit. Especially, the conjugate gradient in the DC unit is determined. The widely used U-net is applied in the learnable regularization module. Besides, η is a learnable regularization parameter. The same regularization module is applied to the generative network, and the weights of the regularization module in different stages are shared. Hence, a significant reduction in network complexity is allowed.
Discriminative network and loss function
The discriminator network $D_{\vartheta _D}$
and the generator network $G_{\vartheta _G}$
are updated alternately to solve the adversarial maximum–minimum problem to achieve the Nash equilibrium, as shown in equation (12). The general idea behind this formulation is that it allows one to train a generative model with the goal of fooling a differentiable discriminator that is trained to distinguish reconstruction images from the ground truth. Equipped with this strategy, the generative network can learn solutions that are highly similar to the ground truth, hence it is difficult to classify with $D_{\vartheta _D}$
. Additionally, to avoid the max-pooling of the entire network, LeakyReLU activation is used. The superscript ground truth is represented by gt:
LM-GAN trains the generation function that estimates its corresponding denoised counterpart for a given rough image. To achieve this, we train a generator network as a feed-forward CNN $G_{\vartheta _G}$
parameterized by $\vartheta _G = {\{ W_{1\colon k}, \;b_{1\colon k}}$
, which is a weighted mixture of multiple loss components. The weights and biases of the generative network are obtained by optimizing a loss function $\ell _{total}^{loss}$
. For training $\bar{\bar{\chi }}_m^{gt}$
, m = 1, …, M with corresponding $\bar{\bar{\chi }}_m^{noise}$
, m = 1, …, M, it resolves as follows:
The loss function $\ell _{total}^{loss}$
is composed of several loss components. To better implement the performance of the generator network, we improve the components of the loss function on the basis of Ledig et al. [Reference Ledig, Theis, Huszár, Caballero, Cunningham, Acosta and Shi26]. We formulate $\ell _{total}^{loss}$
as the weighted sum of a mean square error (MSE) content loss ($\ell _{MSE}^{loss}$
) and an adversarial loss component ($\ell _{Gen}^{loss}$
) as:
$\ell _{MSE}^{loss}$
is between the generator-reconstructed image and ground truth for each pixel are calculated as:
The generative component of LM-GAN is added to the loss function $\ell _{total}^{loss}$
, in addition to the content loss mentioned so far. On all training samples, the concept of the generative loss $\ell _{Gen}^{loss}$
is based on the probabilities of discriminator $D_{\vartheta _D}( G_{\vartheta _G}( \bar{\bar{\chi }}^{noise}) )$
as:
In order to obtain a more suitable gradient, the network minimizes $-{\rm log}\,D_{\vartheta _D}( G_{\vartheta _G}( \bar{\bar{\chi }}^{noise}) )$
instead of ${\rm log}( 1-D_{\vartheta _D}( G_{\vartheta _G}( \bar{\bar{\chi }}^{noise}) ) )$
, where $D_{\vartheta _D}( G_{\vartheta _G}( \bar{\bar{\chi }}^{noise}) )$
is the probability that the reconstructed image $G_{\vartheta _G}( \bar{\bar{\chi }}^{noise})$
is a ground truth.
Numerical results
In the process of reconstructing the relative permittivity from the scattered field, the performance of the proposed network is evaluated. We implement its architecture in MATLAB on a PC equipped with Intel (R) Core (TM) i7-9800X and GeForce RTX 2080Ti. The results under comprehensive data, including cylinders, “Austria” profile and EMNIST datasets [Reference Cohen, Afshar, Tapson and Van Schaik28] are presented.
In the numerical tests, in order to avoid the inverse crime, we consider a DOI of 2 × 2 m2 in size and discretize the domain into 100 × 100 pixels. In the inversion process, the DOI is divided into 64 × 64 pixels. There are 16 line sources and 32 line receivers are equally placed on a circle centered at (0,0) m and with radius 3 and 6 m. The scattered fields from N i incidences are generated numerically by using the moment method and recorded in a matrix $\bar{\bar{E}}^{sca}$
with the size of N r × N i. Then, additive white Gaussian noise $\bar{\bar{n}}$
was added to $\bar{\bar{E}}^{sca}$
. The resulting noise matrix $\bar{\bar{E}}^{sca} + \bar{\bar{n}}$
is regarded as the measured scattered field used to reconstruct the relative permittivity, and the noise level is quantified as $\vert \vert \bar{\bar{n}}\vert \vert _F<{/}ddol>{/}\vert \vert \bar{\bar{E}}^{sca}\vert \vert _F \times 100\%$
. The operating frequency is 400 MHz, unless stated otherwise, and the scatterers are lossless and fall into the range of non-negative contrast a priori information. Finally, the spatial distribution of the permittivity of the scatterer is reconstructed by using BA [Reference Poli, Oliveri and Massa6], SOM [Reference Chen12, Reference Xu, Zhong and Wang13], the backpropagation scheme (BPS) [Reference Wei and Chen29], LMN [Reference Zhou, Ouyang, Li, Liu and Liu22], and LM-GAN, respectively.
In order to evaluate the quality of reconstructed images, the MSE of the relative permittivity is defined as:
Here, $\bar{\bar{\varepsilon }}_r( r_n)$
and $\bar{\bar{\varepsilon }}( r_n)$
denote the reconstructed and ground truth relative permittivity profiles, respectively. N is the number of tests performed.
Comparison under cylinders
In the first example, we select overlapping cylinders and set the relative permittivity of the cylindrical scatterers between 1 and 2. The scattered field is polluted by 0% noise level, 10% noise level, 23% noise level, and 33% noise level additive white Gaussian noises (AWGNs), respectively. We test these configuration profiles on the trained network. In Fig. 3, the reconstructed permittivity profiles of four noise level tests are shown. As the noise level increases (i.e., the signal to noise ratio (SNR) decreases), the quality of reconstruction gradually degrades. When the noise level is 33% (SNR = 10 dB), a clear boundary can still be reconstructed. In order to quantitatively evaluate the performance of the proposed scheme, the test MSE values of these four representative SNRs are further calculated and tabulated in Table 1. The more noise the scattered field contained, the higher the MSE is. Similarly, it indicates that BPS, LMN, and SOM are slightly inferior to LM-GAN. Moreover, each test using LM-GAN is implemented on the computer within 1 s, thus being suitable for real-time reconstruction.
Reconstructed relative permittivity profiles from scattered fields with 0% (noise-free), 10% (SNR = 20 dB), 23% (SNR = 13 dB), and 33% (SNR = 10 dB) AWGNs for BA, SOM, BPS, LMN, and LM-GAN, where the relative permittivity is between 1 and 2. The first row shows the real images for the test and the other rows are the reconstruction results.
MSE value and cost time comparison among BA, SOM, BPS, LMN, and LM-GAN in example 1
Comparison under EMNIST database
In the second example, to study the promotion generalization of the introduced neural network, we thoroughly evaluate the network trained under EMNIST datasets. The proposed network does not recognize and classify the Latin alphabet but quantitatively reconstructs the contour of the scatterer in the Latin alphabet. Compared with the cylinders in the previous example, these profiles are undoubtedly more challenging. One is because these images are not regular. The other is that the relative permittivity of these unknown objects is non-uniformly distributed. Also, the relative permittivity of the scatterers is set between 1 and 2. The size of these letters and digits in the EMNIST database is 28 × 28 pixels. In the forward problem, the size of DoI is 2 × 2 m2, which is discretized into 128 × 128 pixels. In the inverse problem, DoI is discretized into 64 × 64 pixels in order to avoid inverse crime. According to the experimental experience, we randomly select 100 samples in EMNIST datasets to train the proposed network. Each sample is trained for 100 times. Moreover, the training samples used by LM-GAN in the training phase are all noise-free.
There are two representative profiles to be reconstructed, including the English letter G and English letter Y. They are available only during the testing phase. The relative permittivities of scatterers are set between 1 and 2. In practice, the noise level is often unknown. Therefore, AWGNs with 0% (noise-free), 10% (20 dB), 23% (13 dB), and 33% (10 dB) are added to the scattered field to test the trained LM-GAN, respectively. In Fig. 4, the reconstructed permittivity profiles for BA, SOM, BPS, LMN, and the proposed LM-GAN are shown.
Reconstructed relative permittivity profiles of BA, SOM, BPS, LMN, and LM-GAN from scattered fields under various SNRs, where the relative permittivity is between 1 and 2. The first row shows the ground-truth for the test and the other rows are the reconstruction results.
By comparing the reconstruction results of the four methods, it can be observed that the information loss of the reconstructed object increases gradually with the decrease of the SNR. When the noise level is 33% (10 dB), the profile of the scatterer reconstructed by BA and BPS are obviously smaller than the profile of the ground truth, and obvious distortion appears in Fig. 4. However, LM-GAN can significantly reconstruct the profile of the scatterer with preferable accuracy. Furthermore, compared with the LMN, LM-GAN achieves satisfactory results and outperforms it. The reconstruction results can be quantitatively evaluated by considering the MSE of the normalized overall mismatch in all pixels in Tables 2 and 3. It validates the above analysis. The proposed method can minimize the impact of noise on the reconstructed object due to the adversarial learning method, and has good anti-noise performance. It is observed that the imaging quality of SOM is very similar to LM-GAN by evaluating and examining the reconstruction effects of SOM and LM-GAN. Additionally, LMN is a bit inferior to LM-GAN from the perspective of the MSE of reconstruction. Although the reconstruction effects of SOM, LMN, and LM-GAN are satisfactory, the reconstruction time required in SOM is longer, as can be observed from the average reconstruction time listed in Tables 2 and 3. It is worth noting that the fast computational time of LM-GAN enables real-time reconstruction for dielectric profiles.
MSE value and cost time comparison among BA, SOM, BPS, LMN, and LM-GAN about the English letter G
MSE value and cost time comparison among BA, SOM, BPS, LMN, and LM-GAN about the English letter Y
Comparison under “Austria” profile
In the third example, we use “Austria” profile as a benchmark to evaluate the performance of the proposed network. It is well-known in the inverse-scattering field and also is used as a representative test. It consists of a ring and two small disks with the same characteristics. The ring's outer radius is 0.6 m and the inner radius is 0.3 m, respectively. The radius of the two disks is 0.2 m, as observed in the first row of Fig. 5. The relative permittivities of “Austria” profile are set between 1 and 2.
Reconstructed relative permittivity profiles of BA, SOM, BPS, LMN, and LM-GAN from scattered fields under various SNRs, where the relative permittivity is between 1 and 2. The first row shows the ground truth for the test and the other rows are the reconstruction results.
The reconstructed permittivity values of the five methods are visualized in Fig. 5, the SNR are noise-free, 20, 13, and 10 dB. It is noticed that the corresponding reconstructions of BA and BPS have very serious distortion because of the large error. In addition, when the SNR is 10 dB, we find that the shapes of images reconstructed by SOM and LMN are slightly distorted. It can be observed that the proposed LM-GAN can successfully reconstruct the profiles. Although there are large errors in the reconstructed images with a 10 dB, the locations and shapes of profiles can be roughly reconstructed. The MSE values of all tests are included in Table 4. This implies that the proposed scheme has achieved satisfactory results.
MSE value and cost time comparison among BA, SOM, BPS, LMN, and LM-GAN in example 3
Conclusion
This paper proposed a GAN-based fast solution to tackle linear ISPs through alternatively training of generative network and discriminative network in an adversarial way. A connection between GAN and deep iterative solutions was built up to linear ISPs. Numerical results demonstrated that LM-GAN can accurately reconstruct complicated scatterers' electrical performance parameters and profiles under high-noise conditions with very few training samples. It can be comparable to the state-of-the-art nonlinear inversion algorithms. Furthermore, it outperformed the mentioned learning approaches and BA in reconstructing some challenging profiles, in terms of MSE measure and computational cost time.
Huilin Zhou was born in Jiangxi, China, in 1979. He received his Ph.D. degree in space physics from Wuhan University, Wuhan, China, in 2006. His research interests include ultra-wideband radar imaging and radar signal processing. He is currently the director of the Department of Electronic Information Engineering of Nanchang University, a peer reviewer of the National Natural Science Foundation of China, and a member of the China Electronics Society. He is an anonymous reviewer of international SCI journals such as Advanced Robot Systems, Electromagnetic Waves and Applications, and Applied Computing Electromagnetics.
Huimin Zheng was born in 1997 in Jiangxi, China. He is currently pursuing his master's degree in electromagnetic field at Nanchang University. He is mainly engaged in the research of inverse-scattering imaging methods and radar signal processing.
Qiegen Liu received his B.S. degree in applied mathematics from Gannan Normal University, M.S. degree in computation mathematics, and Ph.D. degree in biomedical engineering from Shanghai Jiaotong University. Since 2012, he has been with the School of Information Engineering, Nanchang University, Nanchang, P. R. China, where he is currently an Associate Professor. During 2015–2017, he was also a postdoc in UIUC and the University of Calgary. His current research interest is sparse representations, deep learning and their applications in image processing, computer vision, and MRI reconstruction.
Jian Liu was born in 1995 in Jiangxi, China. He received his master's degree in 2021 from the Department of Electronic Information, School of Information Engineering, Nanchang University. He is currently pursuing his Ph.D. degree at Wuhan University. He is mainly engaged in the research of inverse-scattering imaging methods and radar signal processing.
Yuhao Wang received his Ph.D. degree from Wuhan University, Wuhan, China, in 2006. He was a Visiting Professor with the Department of Electrical Communication Engineering, University of Calgary, Calgary, AB, Canada, in 2008, and the China National Mobile Communication Research Laboratory, Southeast University, Nanjing, China, from 2010 to 2011. He is currently a Professor with the Cognition Sensor Network Laboratory, School of Information Engineering, Nanchang University (NCU), Nanchang, China. He is the Dean of the Information Engineering School and the Artificial Intelligence Industry Institute, NCU, and the Head of Jiangxi Embedded Systems Engineering Research Center. Since 2016, he is an IET Fellow. His current research interests include wideband wireless communication and radar sensing fusion systems, channel measurement and modeling, nonlinear signal processing, smart sensors, image and video processing, machine learning, and visible light communications.
