2.1 Histogram equalisation-based methods

Histogram equalisation (HE) improves image contrast by smoothing and non-linearly stretching the dynamic range of an image histogram (Pizer et al., Reference Pizer, Amburn, Austin, Cromartie, Geselowitz, Greer, ter Haar Romeny, Zimmerman and Zuiderveld1987). This kind of method can be subdivided into global and local HE, i.e., GHE and LHE (Senthilkumaran and Thimmiaraja, Reference Senthilkumaran and Thimmiaraja2014). GHE directly adjusts the overall greyscale of the degraded image according to the global information (Reza, Reference Reza2004). Quadrant dynamic HE (QDHE), proposed by Ooi and Isa (Reference Ooi and Isa2010), obtains better enhanced images without noise amplification and excessive enhancement. However, the GHE-based enhancement method cannot extract detailed information from low-light images. In contrast, LHE could effectively acquire small-scale details in low-light images to achieve local enhancement (Kim et al., Reference Kim, Kim and Hwang2001). Valuchamy and Subramani (Reference Veluchamy and Subramani2019) introduced an efficient enhancement method based on improved adaptive gamma correction and HE. Although the HE-based methods can be implemented efficiently, they do not suppress noise generation and may lead to image distortion (Wang et al., Reference Wang, Wu, Yuan and Gao2020).

2.2 Retinex-based methods

Retinex theory (Land, Reference Land1964) was first introduced by Edwin Land in the 1970s. It states that the physical properties of an object seen by human eyes are independent of the external environment and the illumination of incident light, but only directly related to the reflective properties of this observed object. The classical retinex-based imaging methods include single-scale retinex (SSR), multi-scale retinex (MSR), and MSR with colour restoration (MSRCR) (Choi et al., Reference Choi, Jang, Kim and Kim2008). SSR (Jobson et al., Reference Jobson, Rahman and Woodell1997b) cannot compensate for both details and illumination enhancement at the same time. Although MSR (Lin and Shi, Reference Lin and Shi2014) has been proposed based on SSR, it suffers from edge blurring when enhancing images. MSRCR (Petro et al., Reference Petro, Sbert and Morel2014) can be effective in reducing random noise and restoring image colours. As the correlation between different colour components is neglected in colour restoration, it often suffers from colour inversion and distortion problems (Jobson et al., Reference Jobson, Rahman and Woodell1997a; Ma et al., Reference Ma, Fan, Ni, Zhu and Xiong2017). Kim et al. (Reference Kim, Lee, Park and Lee2019) used the maximum value obtained during diffusion as the illumination component. After adjusting the illumination component by global stretching, it is combined with the reflection component to generate the final enhancement image with local refinement.

2.3 Deep learning-based methods

Compared with traditional imaging methods, deep learning-based low-light image enhancement methods have gained significant attention from both academia and industry. These methods are mainly divided into two classes: supervised and unsupervised learning methods (Zhao and Shi, 2019). The supervised learning methods require training model with labelled paired data to optimise network parameters. For example, Lore et al. (Reference Lore, Akintayo and Sarkar2017) designed a stacked sparse denoising autoencoder to enhance low-light images, demonstrating the effectiveness under different imaging conditions. Cai et al. (Reference Cai, Gu and Zhang2018) trained an enhancer for improving the contrast of underexposed or overexposed images. As these methods require paired low-visibility and normal images for supervised training, there is a higher demand on the datasets. In contrast, the unsupervised or self-supervised learning methods are free from the intrinsic limitation of labelled data. The unsupervised learning-based methods train models based on the loss functions and the light components of degraded images. Guo et al. (Reference Guo, Li, Guo, Loy, Hou, Kwong and Cong2020) proposed to consider the low-light image enhancement as an image-specific curve estimation task. Xu et al. (Reference Xu, Yang, Yin and Lau2020) developed a decomposition-and-enhancement framework in the frequency domain, which obtains the enhancement results by recovering low-frequency information and high-frequency details from degraded images. More recently, a novel multi-branch topology residual block (MTRB) strategy (Lu et al., Reference Lu, Guo, Liu and Ren2022) was proposed to restore low-light images. Benefiting from the development of generative adversarial networks (GAN), Jiang et al. (Reference Jiang, Gong, Liu, Cheng, Fang, Shen, Yang, Zhou and Wang2021) proposed the EnlightenGAN to train the model using unpaired images. However, it is intractable to effectively restore the low-light images directly using existing deep learning methods in maritime scenes.

3.1 Network architecture

We design the LGAN with reference to the attention-guided GAN, as shown in Figure 1. In the generator, the attention network estimates the light distribution of low-light images to distinguish the underexposed and normally exposed regions. The attentional feature maps enable the enhancement network to focus on local information of the degraded images, which can effectively solve the problem of over-enhancement. The enhancement network obtains an enhanced image by performing a series of transformations on the degraded image. It mainly consists of three components, i.e., residual network (Res-Net), PDC module, and feature fusion module (FFM). Res-Net is a residual network with 15 residual blocks for extracting deep features of low-light images. Each residual block contains two convolutions and a shortcut path. The residual blocks learn features by skip connection, which is a flexible structure (Liu et al., Reference Liu, Guo, Lu, Chui and Gupta2022).

Figure 1.

Attention-guided generative adversarial network. It consists of attention network and enhancement network. Further, the enhancement network includes three parts: Res-Net, PDC module and FFM

Although the attention-guided GAN can effectively enhance low-illumination images, the long computational time does not satisfy the requirement of real-time vessel detection. To reduce the computational effort and to realise real-time computation, we propose the lightweight version (i.e., LGAN) based on the attention-guided GAN shown in Figure 1. When designing the LGAN, the attention network was first removed to reduce the number of network parameters. Accordingly, the self-regularised light distribution map is exploited to implement the attention mechanism, which can effectively prevent overexposure while reducing the computational effort. At the same time, the feature extraction network is changed from Res-Net-15 to Res-Net-5 to further simplify the network. Further, we replace the standard convolution in the model with the depthwise separable convolution, and replace the residual block in Res-Net with the inverse residual. To avoid information loss, only the activation function of the last point convolution is retained, and the batch normalisation is removed. The structure of the generator in LGAN is shown in Figure 2. The generator G of LGAN consists of three main parts, i.e., inverted residual network (Inverted Res-Net), PDC module and FFM.

Figure 2.

Flowchart of LGAN. LGAN has three main components, i.e., Inverted Res-Net, PDC module and FFM. In particular, the attention mechanism is implemented by introducing a self-regularised light distribution map in LGAN

The Inverted Res-Net is the backbone of our LGAN, which avoids information loss while deepening the network hierarchy. The addition of the inverse residual module can effectively strengthen the ability of the LGAN to extract features from low-illumination images. The PDC module is applied to multi-scale spatial feature extraction, which minimises the loss of structural information. FFM fuses the deep and multi-scale features extracted by the Inverted Res-Net and PDC module.

The original low-illuminance images and the brightness distribution maps are stitched together as the inputs of LGAN. The inputs first pass through a residual network structure with five inverted residual modules to extract features of the low-light images. Dilated convolution can increase the receptive fields and capture multi-scale context information without increasing the amount of network calculations. We thus add the PDC module after the Inverted Res-Net to extract multi-scale spatial features to avoid or minimise the loss of structural information. The PDC module includes a group of dilated convolutions which have different dilation rates. It has four parallel paths, each of which contains a dilated convolution and a convolution kernel of size 1 × 1. The related dilation rates are one, two, four and six, respectively. The features extracted by Inverted Res-Net are spliced with the features extracted by each path of the PDC module. Four convolutional layers are then exploited to fuse the spliced features. They are multiplied with the illuminance distribution map and then added to the original low-illuminance image for final output.

Normally, there is an inconsistent degree of information loss in different illuminance regions of an image, thus we designed a discriminator, D, which could autonomously give different attention to different illuminance regions. The discriminator D consists of seven convolutional layers and two fully connected layers. In particular, multiple convolutional layers are used progressively to extract image features. The fully connected layers determine whether the image is real based on the extracted features.

The input images of the discriminator D are first passed through five convolutional layers for feature extraction. The feature maps extracted from the fifth convolutional layer pass through two branches. One branch is used to calculate the loss with the illuminance distribution map. The other branch works as input to the sixth convolutional layer. After two convolutional layers and fully connected layers, the extracted features are summarised to obtain the discriminant results. In order to reduce the parameter redundancy, we obtain a lighter GAN model by re-training this discriminator after pruning it.

3.2 Loss function of LGAN

Different optical devices and sensors may cause different non-linear distortions. Although the images are processed when constructing the dataset, there are still varying degrees of pixel-level offset between images. Therefore, it is obviously difficult to obtain accurate outputs using only loss functions based on pixel point differences, such as ${L_1}$ and ${L_2}$ loss functions. To improve the overall perceptual quality of the recovered image, we propose an improved loss function, which mainly consists of the generator loss ${L_G}$ and the discriminator loss ${L_D}$.

3.2.1 Perceptual loss

The VGG-16 network pre-trained on the ImageNet dataset has strong feature extraction capabilities. It is typically applied to extract high-level features from both generated images and target images. The similarity among images is measured by calculating the pixel-level differences between these higher-level features. Therefore, the content perception loss defined in this paper is as follows:

(1)\begin{equation}{L_{\textrm{con}}} = \frac{1}{N}||{{\phi_j}({G(I )} )} - {\phi _j} {({\hat{I}} )} ||_2^2,\end{equation}

where ${\phi _j}$ represents the high-level feature extracted in the j-th convolutional layer of VGG-16, G represents the network of generator, N is the total number of training samples, I and $\hat{I}$ denote the input and target images, respectively.

3.2.2 Colour loss

It is necessary to measure the differences in colour between the enhanced and normally-illuminated images. To better evaluate the colour differences, the Gaussian blur function is employed to weaken edge details in an image (Ignatov et al., Reference Ignatov, Kobyshev, Timofte, Vanhoey and Van Gool2017). In this work, the Euclidean distance between the images after Gaussian blurring is given by

(2)\begin{equation}{L_{\textrm{col}}} = ||{g({G(I )} )} - g {({\hat{I}} )} ||_2^2,\end{equation}

where g denotes the Gaussian blur function.

3.2.3 Adversarial loss

Adversarial loss urges the generator to generate a more natural enhanced image in terms of colour, texture, contrast, etc. Its definition is as follows:

(3)\begin{equation}{L_{adv}} = \log (1 - D({G(I )} )),\end{equation}

where D represents the network of the discriminator.

3.2.4 Gradient loss

As maritime images captured in low-visibility environments tend to be heavily affected by noise, image quality may suffer to some degree. The gradient loss is thus designed to achieve the denoising effect by calculating the square of the gradient difference between images. It makes the enhanced image spatially smooth. The gradient loss can be expressed as

(4)\begin{equation}{L_{gd}} = ||{{\nabla_x}} I - {{\nabla_x}\hat{I}} ||_2^2 + ||{{\nabla_y}} I - {{\nabla_y}\hat{I}} ||_2^2,\end{equation}

where ${\nabla _x}$ and ${\nabla _y}$, respectively, denote the gradients in the x and y directions.

Compared with normal convolutions, deep separable convolutions can reduce the amount of parameters while causing a partial loss of network performance. The ${L_2}$ loss is then applied to ensure structural similarities between the generated and clear images. The ${L_2}$ loss function is defined as follows:

(5)\begin{equation}{L_2} = ||{I^{\prime}} - {\hat{I}} ||_2^2,\end{equation}

where $I^{\prime}$ represents the feature obtained by the Inverted Res-Net.

Therefore, the total loss of generator in LGAN is defined as follows:

(6)\begin{equation}{L_G} = {\omega _{\textrm{con}}}{L_{\textrm{con}}} + {\omega _{\textrm{col}}}{L_{\textrm{col}}} + {\omega _{\textrm{adv}}}{L_{\textrm{adv}}} + \sigma {L_{\textrm{gd}}} + \mu {L_2},\end{equation}

where ${L_{\textrm{con}}}$, ${L_{\textrm{col}}}$, ${L_{\textrm{adv}}}$, ${L_{\textrm{gd}}}$ denote the content perceived loss, colour loss, adversarial loss and gradient difference loss, respectively, ${\omega _{\textrm{con}}}$, ${\omega _{\textrm{col}}}$, and ${\omega _{\textrm{adv}}}$ represent the corresponding weight parameters, $\sigma$ and $\mu$ are weighting parameters.

Based on the distribution of light in different light regions between the enhanced and sharp images, the discriminator D obtains the probability that the input is true or false. Thus, the discriminator loss ${L_D}$ can be written as follows:

(7)\begin{equation}{L_D} ={-} \log (D(\hat{I})) - \log (1 - D(G(I))) + \gamma {L_{\textrm{map}}},\end{equation}

where $\gamma$ denotes the weight parameter, ${L_{\textrm{map}}}$ represents the loss between the feature maps extracted from discriminator and the light distribution maps. The ${L_{\textrm{map}}}$ can be written as follows:

(8)\begin{equation}{L_{\textrm{map}}} = \frac{1}{N}||{D{{(G(I))}_5}} - A ||_2^2 + \frac{1}{N}{||{D(\hat{I})} _5} { - 0} ||_2^2,\end{equation}

where $D{({\cdot} )_5}$ represents the fifth layer convolution of discriminator, A indicates the light distribution map. Since $\hat{I}$ indicates a real clear image, zero means it has no specific region on which to focus.

4.1 Synthetically-degraded image generation

The LGAN is proposed to enhance low-light maritime images. It learns how to map from degraded images to latent sharp images through supervised learning. It thus needs many paired images to complete the network training. However, it is hard to capture both low-visibility and sharp images jointly in a dynamic environment. The existing datasets could not be directly employed to train our network for the enhancement of low-light maritime images.

In this paper, a mixed dataset of realistic and synthetic images is used to complete the network training. The synthetic low-illumination images are obtained by globally reducing the brightness of normal images. The dataset we used is partially derived from the publicly available SeaShips dataset (Shao et al., Reference Shao, Wu, Wang, Du and Li2018). By filtering out poor-quality images (e.g., motion blur and focus blur), 500 normally-illuminated images of size 1920 × 1080 were randomly obtained. Each clear image was translated from the RGB (red, green, blue) colour space to the Hue-Saturation-Value (HSV) colour space. The illumination component V of each image was then reduced by a randomly selected scale factor in the range from 0 to 0 ⋅ 5. After transforming them back into RGB colour space, we finally obtained 500 pairs of clear/low-light images. Figure 3 shows the synthesised results from some clear images.

Figure 3.

Some examples of paired clear/low-light images. The original sharp images are on the top row, above the corresponding synthetic low-light images

To increase the diversity of training data samples, another part of the training dataset was derived from the LOL dataset (Wei et al., Reference Wei, Wang, Yang and Liu2018), which has 500 pairs of low-illumination/clear images acquired from the real world with a resolution size of 600 × 400. To reduce the training time, the training images were first randomly cropped into image blocks of size 128 × 128 before being fed into the network. In addition, the dataset was enlarged using popular augmentation methods, such as random translation and horizontal and vertical flipping. Such operations can effectively strengthen the generalisation ability of an image enhancement network. Results with different augmentation methods are shown in Figure 4.

Figure 4.

Examples of different augmentation methods: (a) original images, and augmented images obtained by (b) horizontal flipping, (c) vertical flipping, (d) random translation

4.2 Implementation details

During the network training, the initial learning rate, decay rate and decay coefficient are set to 0 ⋅ 002, 1,000 and 0 ⋅ 96, respectively. Moreover, the exponential decay method is used to reduce the learning rate. Both Adam optimiser and momentum optimiser are exploited in the generator and discriminator for network parameter optimisation. The LGAN is trained using a small batch training method with a batch size of eight and a number of iterations of 200. The weight parameters ${\omega _{\textrm{con}}}$, ${\omega _{\textrm{col}}}$, ${\omega _{\textrm{adv}}}$, $\gamma$, $\sigma$ and $\mu$ are 2, 11, 0 ⋅ 01, 0 ⋅ 05, 10 and 1, respectively.

In this paper, each convolutional layer adopts the same filling method. Meanwhile, to increase the nonlinearity of the LGAN, only the last convolutional layer of the generator uses the sigmoid function as the activation function. Leaky rectified linear unit (Leaky ReLU) is added after all other convolution layers, and the negative slope of Leaky ReLU is set to 0 ⋅ 2.

To reduce the computing time of the LGAN, the self-regularised light distribution map (Jiang et al., Reference Jiang, Gong, Liu, Cheng, Fang, Shen, Yang, Zhou and Wang2021) can be obtained through the following equation

(9)\begin{equation}A = 1 - \frac{{{{\max }_c}(I)}}{{255}},\end{equation}

where ${\max _c}({\cdot} )$ represents the maximum value of each channel of input I. The light distribution maps obtained by Equation (9) are stitched with the low-visibility image as the inputs to LGAN.

4.3 Comparisons with other competing methods

In this subsection, the following comparison methods are chosen to participate in the image enhancement experiments.

• • SRIE: Simultaneous reflectance and illumination estimation (Fu et al., Reference Fu, Zeng, Huang, Zhang and Ding2016) divides the low-visibility image into reflectance and illumination components. The recovered image could be obtained using the retinex theory. It effectively improves the whole brightness and contrast of low-visibility images.

• • LIME: LIME (Guo et al., Reference Guo, Li and Ling2016) is a simple and efficient enhancement method that refines the initial light map through a prior as the final light map, and generates the result based on the light map.

• • RetinexNet: RetinexNet (Wei et al., Reference Wei, Wang, Yang and Liu2018) constrains the consistency of reflectance for low-light/clear images. Numerous experiments have shown that it can obtain a better low-visibility enhancement effect and represent image decomposition well.

• • DPED: DSLR photo enhancement dataset (DPED) (Ignatov et al., Reference Ignatov, Kobyshev, Timofte, Vanhoey and Van Gool2017) is an end-to-end enhancement method, which can directly generate enhanced images from low-visibility images through the network.

• • LightenNet: LightenNet (Li et al., Reference Li, Guo, Porikli and Pang2018) directly learns the mapping of low-illumination images to light components through the convolutional neural network, then enhances the images based on retinex theory.

4.4 Evaluation metric

To evaluate the performance of different methods, two full reference evaluation metrics, i.e., peak signal-to-noise ratio (PSNR) and structural similarity index measure (SSIM), are employed to evaluate the similarity between images. Moreover, two no-reference metrics, i.e., NIQE and BRISQUE, are chosen to assess the visual effect of generated images.

• • PSNR: PSNR (Wang and Bovik, Reference Wang and Bovik2009) is an objective measure of the level of image distortion or noise. The higher its value, the less noise the image contains. The formula for calculating PSNR is as follows:

  (10)\begin{equation}\textrm{PSNR}({\hat{Y},Y} )= 10\log \frac{{{M^2}}}{{\textrm{MSE}({\hat{Y},Y} )}},\end{equation}

where M denotes the maximum pixel value, and $\textrm{MSE}({\cdot} )$ indicates the mean square error.

• • SSIM: SSIM (Wang et al., Reference Wang, Bovik, Sheikh and Simoncelli2004) is used to measure the brightness, contrast and structural similarities between two images. Therefore, SSIM displays favourable agreement with human observers. Its value has a range from 0 to 1. The closer the value is to 1, the more similar the two images are.

• • NIQE: NIQE (Mittal et al., Reference Mittal, Soundararajan and Bovik2012b) does not need to adopt a reference image, but simply uses the measurable deviation of the target image to obtain the image quality evaluation value according to the natural statistical characteristics. The NIQE metric is calculated as follows:

  (11)\begin{equation}({{v_1},{v_2},{\Sigma _1},{\Sigma _2}} )= \sqrt {{{({{v_1} - {v_2}} )}^\textrm{T}}{{\left( {\frac{{{\Sigma _1} + {\Sigma _2}}}{2}} \right)}^{ - 1}}({{v_1} - {v_2}} )} ,\end{equation}

where ${v_1}$, $\textrm{v_2}$ and ${\Sigma _1}$, $\Sigma _2$ denote the mean vectors and covariance matrices of the Gaussian distribution models of the natural and enhanced images, respectively.

• • BRISQUE: BRISQUE (Mittal et al., Reference Mittal, Moorthy and Bovik2012a) is a reference-free spatial domain image quality evaluation algorithm. The principle of BRISQUE is to extract local normalisation coefficients (MSCN) from images to fit them to generalised Gaussian distribution and asymmetric generalised Gaussian distribution. In addition, it extracts 36 features using multi-scale images and derives objective quality scores using support vector machine.

4.5 Experimental results on syntactic maritime images

Figures 5–7 visualise the enhancement results of LGAN and other comparable methods on synthetic low-illumination images. It can be seen that SRIE and DPED have limited ability to enhance images with dark regions. Their results have lower overall brightness compared with other enhanced images. The images produced by RetinexNet have obvious differences in structure and colour compared with the sharp images. LightenNet has limited effect on image enhancement, whose restored images have irregular colour spots. As shown in Figure 7, LIME has better visual effect, but suffers from colour distortion. In contrast, the proposed LGAN has a more stable enhancement effect on images with different illumination levels. This is mainly due to the incorporation of attention mechanism, which enables the LGAN to perform adaptive enhancement and have powerful generalisation ability.

Figure 5.

Visual comparisons of proposed LGAN and mainstream methods on Image 1: (a) synthetic low-light image, outputs obtained by (b) SRIE, (c) LIME, (d) RetinexNet, (e) DPED, (f) LightenNet, (g) LGAN, (h) ground truth (GT)

Figure 6.

Visual comparisons of proposed LGAN and mainstream methods on Image 2: (a) synthetic low-light image, outputs obtained by (b) SRIE, (c) LIME, (d) RetinexNet, (e) DPED, (f) LightenNet, (g) LGAN, (h) ground truth (GT)

Figure 7.

Visual comparisons of proposed LGAN and mainstream methods on Image 3: (a) synthetic low-light image, outputs obtained by (b) SRIE, (c) LIME, (d) RetinexNet, (e) DPED, (f) LightenNet, (g) LGAN, (h) ground truth (GT)

PSNR and SSIM are used to further verify the enhancement ability and structure preservation ability of LGAN. From Table 1, it can be seen that LGAN performs well in PSNR and SSIM evaluation metrics compared with others. It indicates that LGAN not only significantly improve the contrast of images and retain more detailed information, but also suppress unwanted noise. Thus, the visual qualities of low-light images are significantly improved accordingly.

Table 1.

Comparisons of PSNR and SSIM values for different competing methods on Images 1–3

4.6 Experimental results on realistic maritime images

To further verify the superiority of LGAN for low-light maritime image enhancement, numerous experiments were conducted on realistic low-illumination images. The comparisons of visual effects from different methods are shown in Figure 8. LIME can effectively improve the brightness of low-light images. However, the enhanced results often suffer from the problem of local overexposure. The images enhanced by LIME are not natural enough in practice. RetinexNet can cause colour distortion in the images which is not suitable for auxiliary vessel detection. The images generated by LightenNet have trouble with unnatural colour transitions. Although the images produced by DPED and SRIE are visually pleasing, they both have a certain level of detail loss. Compared with LGAN, the enhanced images of SRIE are more blurred, which indicates that SRIE is less effective in recovering details. Although DPED can achieve the purpose of improving illumination, the enhanced images show colour deviation and colour distortion. In contrast, the LGAN is more capable of enhancing the brightness and contrast, while being able to retain more image details. The images enhanced by LGAN are more natural and have better visual appearances.

Figure 8.

Visual comparisons of proposed LGAN and mainstream methods on Images 4–6. The three rows, from top to bottom, are Image 4, Image 5 and Image 6, respectively

In addition, NIQE and BRISQUE were selected to evaluate quantitatively the imaging quality for different image enhancement methods. As illustrated in Table 2, although the NIQE score of LGAN is lower than DPED in Image 5, the BRISQUE metrics of LGAN perform better than other competing methods. It shows that LGAN performs better enhanced effects on low-illumination maritime images.

Table 2.

Comparisons of NIQE and BRISQUE values for different competing methods on Images 4–6

4.7 Running time analysis

To verify the advantages of LGAN in terms of model size and running time, this subsection compares the proposed LGAN method with three other methods, i.e., RetinexNet, DPED and LightenNet. As shown in Table 3, two maritime images with different resolutions were chosen to compare the running time for different image enhancement methods. All experiments were performed on a computer with the Intel (R) Core (TM) i5-10600KF CPU @ 4.10 GHz and Nvidia GeForce GTX 2080 Ti GPU.

Table 3.

Comparisons of running time (in seconds) and model size for different competing methods

Although the model size of the LGAN is slightly larger than LightenNet, LGAN generates shorter running time than both DPED and LightenNet. This is because only standard convolution is used in DPED for feature extraction. LightenNet uses an additional intermediate operation such as bootstrap filtering in the image enhancement process, which consumes some extra time. In contrast, LGAN uses deeply separable convolution instead of standard convolution to significantly reduce computational effort. In addition, the model pruning and compression operations allow the model size to be greatly compressed without compromising the imaging performance. These operations allow LGAN to satisfy the demands of real-time low-light image enhancement in maritime surveillance tasks.

4.8 Experiment on vessel detection under low-light environment

This subsection presents vessel detection experiments designed to demonstrate that the proposed LGAN can improve the robustness and accuracy of maritime target detection in low-light imaging environments. In the current literature, many target detection methods have been proposed to promote traffic situation awareness. In this paper, the popular YOLOv4 (Bochkovskiy et al., Reference Bochkovskiy, Wang and Liao2020) is used to perform the vessel detection experiments. The YOLOv4 model employed in our experiments is trained using the SeaShips dataset. We adopt two main vessel detection schemes. One solution directly uses YOLOv4 to detect vessels on low-light images. The other solution first uses the LGAN to recover degraded images, and then performs the YOLOv4-based vessel detection.

The vessel detection results on synthetic and realistic scenarios are shown in Figures 9 and 10, respectively. It is obvious that the implementation of YOLOv4 alone in a low-illumination environment is prone to missed and undetectable detection. If the proposed LGAN is first exploited to perform low-light image enhancement, the intrinsic features of targets of interest could be highlighted from low-light environments. Therefore, the robustness and accuracy of vessel detection are significantly improved accordingly.

Figure 9.

Vessel detection experiments on synthetic low-visibility maritime images (a). Vessel targets are not easily detected in the synthetic low-illumination images. After enhancement by LGAN (b), the target detection results are very close to ground truth (GT) (c)

Figure 10.

Vessel detection experiments on real maritime images (a). Some vessels are not detected on real low-visibility images and there is a problem of repeated detection. In contrast, using LGAN to pre-process low-visibility images can significantly improve the robustness and accuracy of vessel detection (b)