3.1.1 Areas with small latitude and longitude range

Chapter 30 of the ‘German Sea Navigable Waters Traffic Regulations’ (Federal Maritime and Hydrographic Agency, Reference German Traffic Rec2023) stipulates that the sailing standard for oil tankers with a load of more than 2,000 tons is the range of visibility more than 1000 m. Additionally, the sailing standard for oil tankers with a load of less than 2,000 tons is greater than 500 m. Paragraph 14$\cdot$2$\cdot$3 of the Prohibited and Restricted Traffic Regulations in the ‘German Maritime Navigable Waters Regulations’ stipulates that for ships entering and leaving the port, the ship traffic service may individually authorise them to exempt them from the prohibited traffic restrictions, but must satisfy the condition that the visibility distance exceeds 500 m.

The port of Rotterdam in the Netherlands implements hierarchical management and pilotage is mandatory below 700 m line of sight (including vessels with pilot exemption certificates). When visibility at any point in the port of Rotterdam falls below 700 m, the Ministry of Vessel Traffic and Operations will enforce the following rules: (1) if the visibility is less than 200 m, the VTMS centre will communicate with the pilot to see if it is possible to sail; (2) if the visibility is between 200 and 500 m, a tug escort will be arranged according to the vessel's needs, and the VTMS centre will consult with the pilot and the tug to see if it is possible to sail; (3) if the visibility is greater than 500 m, vessels with a draft between 17$\cdot$4 and 22$\cdot$55 m are prohibited from entering the port; (4) if the visibility in the inland river is less than 1,000 m or the visibility in the port entry area is less than 2,000 m, the traffic management department will provide navigational guidance through the VTMS.

The ‘Guidelines for Vessel Navigation during Restricted Visibility’ (Harbormasters Office Auckland, Reference Auckland Council2010) formulated by the Port of Auckland, New Zealand, clearly require the Port of Auckland to establish fog navigation procedures applicable to the port, which include: pre-establishing vessel routes, pre-entry notification for fog navigation, call program. When passenger ships, ferries, ro-ro passenger ships, etc. encounter foggy weather, the master must assess the situation before making a decision. If the master deems it necessary to take action in the event of poor visibility, a request should immediately be sent to the port dispatcher. Upon receipt of the request, the dispatcher will issue a notice to all vessels requiring all vessels to implement response measures in the event of poor visibility. Action will be taken and a message will be sent via the VTMS centre to inform the VTMS contact person that a particular area of the port needs to take action in the event of poor visibility. When passenger ships, ferries, ro-ro passenger ships, etc., sailing in an area where poor visibility countermeasures are required, find that visibility has improved, the master should make an early decision as to whether poor visibility countermeasures are still required. If, at that time, the master is of the opinion that it is no longer necessary to implement the degraded visibility countermeasures when visibility improves, the master on duty should immediately send a report to the dispatcher and request that the procedure be cancelled. When making a decision, the master should ensure that all areas of port navigation are not affected by the poor visibility. If some areas are still affected, the procedure cannot be cancelled. With regards to speed regulations, the port has the following requirements: ships must maintain a safe speed in accordance with the requirements of the international rules for the avoidance of collisions. When navigating in or near waters with poor visibility, ships should be able to stop at a distance of less than half the visibility distance.

3.1.2 Areas with large latitude and longitude range

(1) China

China has done a lot of research and exploration on fog shipping technology. At present, China has established a complete fog shipping management system, including fog zone delineation, forecasting and early warning, channel marking, communication and navigation. The China Maritime Safety Administration (MSA) divides domestic sea areas into different fog zones based on factors such as density of maritime traffic, frequency of maritime accidents and meteorological conditions. The division of fog zones is mainly to facilitate the navigation of ships in foggy weather and to improve the safety of maritime traffic. The division criteria are mainly based on factors such as the density of maritime traffic, the frequency of maritime accidents and meteorological conditions, and divide domestic sea areas into different fog zones. The specific classification criteria are as follows. (1) Maritime traffic density: the higher the traffic density, the greater the possibility of being divided into fog areas. (2) Frequency of marine accidents: in some sea areas, accidents are more likely to occur due to factors such as terrain and climate. These sea areas are also prone to turning into fog areas. (3) Meteorological conditions: meteorological conditions are an important basis for the division of fog areas. For example, meteorological conditions such as both relative humidity of more than 85% and visibility of less than 1 nautical mile are important criteria used to divide fog areas.

According to the classification standards, the China MSA divides the domestic sea areas into several fog areas, including the East China Sea fog area, the South China Sea fog area and the Bohai Sea fog area. Different fog areas have different characteristics. For example, fog areas in the East China Sea are concentrated from March to June, and fog is most frequent along the west coast of the Taiwan Strait and in the waters near the Zhoushan Islands; sea fog in the South China Sea is limited to the coast of China. From February to June, the fog areas are more concentrated along the Qiongzhou Strait and Leizhou Peninsula; the fog area in the Bohai Sea is mainly concentrated in the east of Bohai Bay (Huang et al., Reference Huang, Gao and Shi2022). The foggy area increases from south to north. From South China Sea, East China Sea to Bohai Sea, the time of sea fog is continuously delayed. The China MSA of each sea area provides appropriate information to vessels in the fog area according to the characteristics of the local sea fog, including channel marking information, foggy weather forecast information, navigation guidance and emergency response to foggy weather emergencies.

1. (1) The East China Sea fog area

Shanghai MSA's 2019 regulations on poor visibility in the waters of the Yangtze River Estuary in Shanghai Port are as follows. When the visibility distance of the deep-water channel is less than 1,000 m, ships that have not entered the channel are prohibited from entering, and ships that have entered the channel should navigate cautiously. Ships should stop operations and give way to the channel, and ships are not allowed to use the slope to rendezvous. The regulations on the poor visibility of the Huangpu River section of the route are as follows: when the visibility is less than 1,000 m, ships should sail slowly; when the length is less than 500 m, large ships are prohibited from sailing (large ships refer to motor ships with a gross ton of more than 1,600 tons or non-motorised ships with a load capacity of more than 1,000 tons); when the length is less than 100 m, all types of ships are prohibited from sailing.

The notice issued by the Xiamen MSA on ‘further strengthening the supervision and management of navigation safety during the foggy season’ stipulates that if the visibility in the sea area of the port is less than 1 nautical mile and continues to deteriorate, the port management department, pilotage unit and vessels should be prepared to suspend navigation; if visibility continues to deteriorate, large vessels and vessels carrying dangerous goods are banned from sailing when visibility is less than 1,200 m; small and medium-sized cargo vessels are banned from sailing when visibility is less than 1,000 m; and all vessels are banned when visibility is less than 700 m.

1. (2) The South China Sea fog area

The ‘Regulations on the Safe Navigation of Ships in the Areas under the jurisdiction of the Guangdong Maritime Safety Administration’ (Guangdong Maritime Safety Administration, 2021) stipulates the following restrictions on the navigation of ships in water areas under poor visibility weather conditions: (1) when the navigation visibility is less than 2,000 m, in the waters from the Guishan Pilot Anchorage to the Sampan Island, the speed of the ships shall not exceed 10 knots; in the waters from Sampan Island to Huangpu, the speed of ships shall not exceed 8 knots, except for high-speed passenger ships; (2) when the visibility is less than 1,000 m, ships in waters with limited visibility are prohibited from leaving the dock or anchorage, except for high-speed passenger ships; (3) when the visibility is less than 500 m, high-speed passenger ships are not allowed to sail away from the docks or anchorages in waters with limited visibility.

The Hong Kong Marine Department regularly issues and updates the ‘Safety of Navigation in Impaired Visibility Notice’ (Marine department Hong-Kong, 2012), which requires ships to strictly comply with the ‘International Regulations for Preventing Collisions at Sea 1972’ when navigating in Hong Kong waters, especially in conditions of poor visibility. All vessels navigating in Hong Kong waters must maintain a safe speed and not exceed the applicable speed limit. When navigating in fog, heavy rain or reduced visibility, extreme care shall be taken and appropriate signals shall be given in accordance with ‘the International Regulations for Preventing Collisions at Sea, 1972’. The Marine Department has established a Vessel Traffic Monitoring Centre to provide traffic monitoring services to vessels in Hong Kong waters. It mainly uses an advanced digital radar network with computer system, VHF communication equipment and database to control the entering and leaving port activities. The radar surveillance area covers the entire waters in which Hong Kong vessels operate. In addition, the Traffic Control Substation at Kwai Chung Container Terminal No. 8 is responsible for providing 24-hour navigation services to vessels in Kwai Chung Wharf Harbor to enhance the safety and efficiency of berthing vessels in the harbour.

1. (3) The Bohai Sea fog area

Tianjin MSA's ‘Tianjin Maritime Safety Administration Ship Traffic Management System Safety Supervision and Management Rules’ (Tianjin Maritime Safety Administration, 2023) stipulates that the main channel of Tianjin Port shall implement one-way navigation when the visual range is less than 3,000 m; when the visual range is less than 1,000 m, it is a ‘small-boat channel’, ships are prohibited from entering and leaving the port; LNG ships with a visual range of less than 1,000 m and a ship capacity of 40,000 m$^3$ or more are prohibited from entering or leaving the port when the visual range at sea is less than 2,000 m.

(2) United States of America

Both the Port of San Francisco and the Port of Oakland are located in San Francisco Bay on the west coast of the United States, so the fog navigation rules are similar in both places. The VTMS Centre will provide suggestions, advice and warnings to the vessel according to the situation, but the actual safety is commercial and the captain as well as the shipping company are fully responsible. The Port VTMS System provides traffic organisation services for motor vessels of 1,600 gross tons and above in its waters. The port requires vessels of more than 300 gross tons, motor vessels of more than 40 m in length, tugs of more than 8 m in length and passenger vessels with a capacity of more than 50 passengers to be authorised to participate in the VTMS Reporting System. Motor vessels of more than 20 m in length, vessels of more than 100 gross tons, tugs of more than 8 m in tow, dredgers and floating craft must be equipped with communication equipment on the bridge.

The Port of Los Angeles and the Port of Long Beach are only 6 miles apart. Their weather conditions are similar, so they share a set of fog navigation rules. The Ports do not have strict requirements for small vessels such as pleasure boats, commercial fishing boats and sport fishing boats to only provide sailing advice. When visibility is between 1 and 0$\cdot$5 nautical miles, ships over 150,000 deadweight are prohibited from sailing; when visibility is less than 0$\cdot$5 nautical miles, no ships are allowed to sail (The Los Angeles/Long Beach Harbor Safety Committee, 2022).

To facilitate comparison of rules in different regions, we have summarised the above content in Table 1.

Table 1.

Waters Traffic Regulations

3.2.1 Fog warning

Sea fog is mainly influenced by factors such as sea air temperature (Gao et al., Reference Gao, Gao and Yang2018b), humidity (Lewis, Reference Lewis2004), air pressure (Kim and Yum, Reference Kim and Yum2012), wind direction (Hou and Wang, Reference Hou and Wang2004), wind speed (Li et al., Reference Li, Wang, Shang and TianYi2012), ocean currents and topography. Sea fog has the characteristics of regionality, large spatial scale ($10^{-7}$–$10^6$ m) and large temporal range (several hours to days), which makes it difficult to accurately predict sea fog on a large scale. However, numerical prediction of regional, medium and small-scale sea fog has some feasibility. Sea fog observation and early warning mainly use data collected by airborne instruments, remote sensing platforms, land-based sensors and other equipment, which are used as boundary conditions or physical parameters and fed into the model for prediction. Forecasting models for sea fog are based on two main methods: statistical forecasting and dynamic forecasting. The former is based on various statistical analysis techniques and the latter is based on physical equations related to fog phenomena.

In 2004, Lewis et al. (Reference Lewis, Koračin and Redmond2004) developed a 24-hour early warning model for sea fog in the Yellow Sea based on the meteorological data of the past ten years using a tree regression model. In 2011, Huang et al. (Reference Huang, Liu, Jiang and Huang2011) used the real-time meteorological information provided by the National Weather Service of the United States as the physical parameters to predict sea fog. This method not only considers the computational error of the model, but also includes the local climate characteristics, and achieves the 24-hour forecast accuracy rate up to 78% in the South China Sea. In 2018, the Fujian Meteorological Bureau deployed a LoRa (Wu et al., Reference Wu, Yu, Ren and Jing2022) sea fog forecasting system with a maximum coverage of 6,000 km$^2$, relying on land-based weather stations and sea floats. The LoRa Sea Fog System has a time resolution of 5 minutes, a spatial resolution of 500 m and an accuracy of 85$\cdot$5%.

With the continuous development of deep learning technology, some researchers have used various tools for prediction by taking advantage of deep learning to extract high-dimensional statistical information. In the same year, Guijo-Rubio et al. (Reference Guijo-Rubio, Gutiérrez, Casanova-Mateo, Sanz-Justo, Salcedo-Sanz and Hervás-Martínez2018) used a sliding dynamic window to improve the ordinal scorer for prediction, which improved the problem of poor prediction accuracy over time. In 2020, Lu et al. (Reference Lu, Zheng and Yang2020) used a small amount of data to complete the TCN_TL network training using time-based convolutional network TCN and transfer learning, and conducted experimental verification in the Qiongzhou Strait. As shown in Figure 1, the 24-hour prediction accuracy rate has increased by 11% compared with before transfer learning.

In 2022, Park et al. (Reference Park, Lee and Jo2022) used a multimodal learning approach that combined temporal and spatial information to predict foggy visibility and successfully forecasted the visibility in the waters off South Korea with an accuracy of 88$\cdot$6% one hour later through experimentation. In 2023, Xiang et al. (Reference Xiang, Zhang and Wang2023) introduced a method for time-lag correlation analysis in deep learning to assess the time-lag effect of foggy weather forecasting. This method allows for sea fog prediction up to 60 hours in advance. After comparison with the method developed by the Forecasting System Laboratory (FSL) of the National Oceanic and Atmospheric Administration (NOAA) in the United States, it was found that the method has a higher false alarm rate for sea fog forecasts with visibility of less than 1 km. The method was found to have fewer false alarms in predicting sea spray with visibility below 1 km.

The dynamic prediction of sea fog based on physical models has been studied since World War II. The numerical prediction of sea fog is mainly based on the atmospheric boundary layer model established by Estoque (Reference Estoque1962) to solve the diffusion equation. According to different simulation dimensions, it is divided into one-dimensional, two-dimensional and three-dimensional simulation. In 1963, Fisher and Caplan (Reference Fisher and Caplan1963) first used one-dimensional simulation to simulate fog, proving the feasibility of numerical simulation and prediction of sea fog. In 1989, Findlater et al. (Reference Findlater, Roach and McHugh1989) used a one-dimensional simulation to make a simple forecast of sea fog on the north-east coast of Scotland. As this simulation only considered the advection and turbulence of the fog, the accuracy was low. In 1997, Hu and Zhou (Reference Hu and Zhou1997) used a two-dimensional model combined with a time-varying humidity equation to study the formation process of sea fog, and pointed out the effects of long-wave radiation cooling and turbulent cooling on the formation of sea fog. The formation of sea fog is affected by many factors, but the one- and two-dimensional simulations are too simplified to simulate the sea fog formation process accurately enough. Therefore, more 3D simulations are used to study the sea fog formation process. Stoelinga and Warner (Reference Stoelinga and Warner1999) used the results of MM5 numerical model simulation to establish an empirical formula for calculating visibility, which played a very good role in promoting numerical simulation research and forecasting of sea fog. Fu et al. (Reference Fu, Zhang and Duan2004) used an improved atmospheric boundary layer model to conduct a three-dimensional simulation study of sea fog in the Yellow Sea. They better simulated the actual sea fog generation and dissipation process in the Yellow Sea and analysed the spatial-temporal three-dimensional distribution characteristics of the sea fog generation and dissipation process. The simulated horizontal visibility distribution is in good agreement with the observation results of satellite cloud images.

3.2.2 Assisted perception of navigation environment

Sea surface images have the characteristics of simple background, large depth of field, single colour and relatively small proportion of sea targets. Sea fog is a common climatic phenomenon in the marine environment, which has a major impact on maritime navigation and activities. Visibility is low in foggy conditions and it is difficult for the crew to fully observe the environment around the ship while it is sailing. To improve the safety of maritime navigation, researchers have developed various fog navigation aids using different devices and algorithms. Fog navigation auxiliary perception technology mainly assists the crew to perform image enhancement, identification, monitoring and other tasks by using existing AIS, radar, infrared imager and other equipment as well as various perception algorithms. This type of research is mainly based on intelligent algorithms combined with radar and infrared waves or processed separately in the back-end.

(1) Radar-based auxiliary detection technology

Radar, which is one of the main means of detecting targets on a ship, detects targets by emitting electromagnetic waves. It is not limited to day or night, and can penetrate rain and fog. In 2023, Duarte et al. (Reference Duarte, Naranjo, López and Del Campo2005) used CWLFM technology to transmit pulse signals that can analyse position and velocity information according to the frequency change of the received signal. In 2021, Zheng et al. (Reference Zheng, Yang, Xie, Song and Xu2020) used Doppler continuous wave radar and compressed sensing with adaptive threshold technology to improve the detection performance of targets and reduce the complexity of information processing. Considering that vehicle-borne radar is relatively mature and widely used, in 2019, Stateczny et al. (Reference Stateczny, Kazimierski, Burdziakowski, Motyl and Wisniewska2019) applied vehicle-borne radar to unmanned vessels to detect obstacles when unmanned vessels enter ports, and verified the feasibility of this method. However, this method can only detect obstacles within a small range, cannot determine the shape of obstacles and requires the support of complex algorithms. As shown in Figure 2, in 2020, Fan et al. (Reference Fan, Cui, Huang and Han2020) used a wavelet transform to extract their respective feature information from radar and visible light based on fused radar and visible light images, and then fused the two by a specific algorithm to generate a new fused image for detection, which achieves high accuracy and robustness.

Figure 2.

Prediction results of TCN and TCN_TL at (a) 24, (b) 48, (c) 72 and (d) 96 hours

Since fog droplets have a scattering and depolarising effect on electromagnetic waves, the radar receives special signals when sea fog is present, from which information such as the location, intensity and linear depolarisation ratio of the sea fog can be analysed. Therefore, radar can be used to detect sea fog in canals. In 2001, Hair et al. (Reference Hair, Caldwell, Krueger and She2001) used a high spectral resolution radar with an iodine vapour light filter to measure fog droplet distribution. In 2005, Xie et al. (Reference Xie, Han and Zhou2005) used a Raman scattering lidar to measure the atmospheric horizontal visibility on fog. In 2019, the atmospheric horizontal visibility measurement system developed by Liao et al. (Reference Liao, Liu, Li and Zhan2019) reduced the error value from 3 km to approximately 0$\cdot$8 km. In 2021, Rong and Lee (Reference Rong and Lee2021) miniaturised the entire radar equipment, significantly reducing the cost of radar sea fog early warning. Lu et al. (Reference Lu, Zheng and Yang2020) used scanning lidar with a wavelength of 532 nm to realise sea fog early warning with a visibility of 5 km. The maximum detection distance is more than 6 km during the day and more than 10 km at night. In 2012, Sen et al., Reference Sen, Yuzhai, Yi, Baosen and Shiliang2013 at the Harbin Institute of Technology (Sen et al., Reference Sen, Yuzhai, Yi, Baosen and Shiliang2013) completed a foggy target imaging experiment using range-gated lidar that was able to clearly image targets behind fog. In 2018, Guy Satat from MIT (Satat et al., Reference Satat, Tancik and Raskar2018) pointed out that for Gm-APD laser lidar, the backscattered laser echo time always shows a very specific distribution pattern, no matter how thick the fog is. Fitting and optimisation algorithms are used to effectively extract the target buried in the noise and then reconstruct the target image. The imaging effect is shown in Figure 3.

Figure 3.

Comparison before and after image fusion

Based on this principle, in 2020, Sang et al. (Reference Sang, Tsai and Yu2020) used single-photon avalanche diode lidar to achieve target imaging detection under low signal-to-noise ratio. Figure 4 shows the recognition effect of dehazing target contour.

Figure 4.

Comparison of reconstructed images in foggy weather

In the same year, Gao et al. (Reference Gao, Sun, Jiang, Liu, Zhou and Fu2018) of the Harbin Institute of Technology used the backscatter of the Gm-APD lidar to reconstruct foggy target images. As shown in Figure 5, Zhang et al. (Reference Zhang, Jia and Zhu2018) take advantage of SNSPD's high detection efficiency and low dark count characteristics to improve the imaging distance of sea fog and extend the measurement range to approximately 180 km. However, this principle relies too much on the pre-set model, and the error is big when the environmental conditions do not match the model.

Figure 5.

Target contour detection before and after dehazing

Millimetre wave radar has greater fog penetration capabilities due to its shorter wavelength and higher energy. Millimetre wave radar imaging systems are diverse and can combine with single focus receivers, raster scans, receiver arrays, antenna arrays, etc. However, the current imaging distance of millimetre wave technology is relatively short, and the imaging quality is difficult to guarantee as the distance increases. The 23rd Institute of the Second Academy of China Aerospace Science and Industry Corporation uses solid-state coherent pulse and continuous-wave dual-mode radar to guide ships into harbours and for coastal navigation at night in foggy weather. It can detect floating objects more than 25 m away within 20 nautical miles, and the peak power of the radar has also been reduced from 200 watts to 1 watt, effectively reducing radiation. As shown in Figure 6, Cao et al. (Reference Cao, Liu and Gao2020) proposed a method to identify marine obstacles and millimetre-wave radar sensor information without joint calibration, which improved the efficiency of obstacle target detection in fog navigation.

Figure 6.

Sea fog target measurement results

In addition, Huang et al. (Reference Huang, Wang, Wen and Peng2023), Sun et al. (Reference Sun, Wang, Cai and Bai2023) and Xing et al. (Reference Xing, Cui, Wang, Xiong and Jiang2017, Reference Xing, Cui, Wang, Xiong and Jiang2024) have focused on detection technology in challenging conditions. As shown in Figure 7, Xing et al. (Reference Xing, Cui, Wang, Xiong and Jiang2024) proposed a feature sharing fusion method, which used different sampling rates to obtain target texture features for judgment, thereby improving the judgment accuracy without affecting the inference speed. In 2017, Wang's team (Xing et al., Reference Xing, Cui, Wang, Xiong and Jiang2017) proposed a fusion system based on radar and a single camera to detect and identify ships in lakes. The method was evaluated in parallel on two sensors, using a collaborative technique to integrate information from lidar and monocular cameras. The radar and camera are used to collect information, and the image plane after radar acquisition and processing is synchronised in space and time with the camera to determine whether the image detected by the radar can be compared with the Support Vector Machine (SVM) classifier and visual significance. The detection results overlap. If the detection results are consistent, the candidate container is determined as the target container. Otherwise, the radar and camera will continuously detect 10 frames again. The radar target projected to the image plane will be compared with the SVM classifier if it is not less than 5 frames. If the visual detection results are consistent, the target can also be determined to be the target to be tracked. If neither of the above two results exists, you need to judge the image of the next frame. At the same time, in this method, Kalman filtering is used to track the target after target verification, and online tracking is used in visual detection. Experimental results show that this method has a high detection rate in ship detection and classification, and the accuracy rate can reach 95$\cdot$68%. However, the effective detection distance of this system is 20 m, which results in low utilisation of the two sensors for monitoring the surrounding environment, which is a defect that needs to be improved.

Figure 7.

Detect and identify obstacles

(2) Auxiliary imaging technology based on infrared camera

Infrared imaging, due to its wavelength characteristics, can be used in any environment with thermal infrared radiation and can effectively deal with foggy environments. Therefore, the use of infrared cameras on ships is also a common defogging imaging method. This method mainly refers to the principle of visible light defogging imaging. In 2020, the Korean Academy of Science and Technology successfully used infrared cameras on the world's first unmanned bulk carrier to avoid obstacles in fog, and combined it with its own ship Internet of Things (IoT) platform to measure the speed of moving obstacles. Because infrared imaging was first used in specialised fields, it has shortcomings such as high noise, low contrast and high non-uniformity. Yin Yisong (Su et al., Reference Su, Yan and Yin2016), Zhou Na (Su et al.) and Yan Mei (Su et al., Reference Su, Yan and Yin2016) combined the panoramic vision system to study the detection methods of distant targets, small targets, as well as weak and small targets at sea. It is practical to use the panoramic vision system to construct a panoramic image of the sea, including dehazing, sharpening, sea line extraction, and distant weak and small target detection. Scientists from many countries have tried to apply the histogram equalisation algorithm directly to the field of infrared image defogging. The processing results show that although the brightness of the image has been increased, the resolution is low, grey scale jumps are prone to distortion and the original image details are blurred or even disappear (Xing et al., Reference Xing, Zhang and Yang2020). In response to such problems, Wang et al. (Reference Wang, Dong, Zhao, Houde and Ji2015) proposed a new target detection strategy based on regional segmentation and wavelet cross-correlation through feature analysis of maritime infrared images. The algorithm first uses the adaptive thresholding method to divide the original image into different regions, then uses wavelet thresholding to suppress the noise in the sub-bands, and finally extracts the real targets based on the inter-sub-band correlation and local singularities in the original image. As shown in Figure 8,it is verified through experiments that this algorithm can effectively overcome the uneven brightness and background interference in fog, and achieve accurate infrared maritime target detection.

Figure 8.

Comparison of test results before and after

Silverman (Reference Silverman2020) proposed histogram projection and platform histogram equalisation algorithms. By selecting an appropriate platform threshold, the platform histogram equalisation algorithm was used to correct the histogram of the original image to limit the pixels with high probability of grey density. This method can effectively compensate for the lack of histogram equalisation and alleviate the phenomenon of grey-scale merging. In addition, given the effectiveness of the Retinex algorithm in processing visible light images, scholars have attempted to apply the Retinex algorithm to the processing of infrared hazy images. For example, Wang et al. (Reference Wang, Fan, Li, Min, Zhang and Chen2023) used the Retinex algorithm and gamma correction to estimate the incident light and improve the brightness of the image, and then corrected the overall colour of the image according to the colour of the incident light to make it suitable for the atmospheric scattering model, and finally used the dark channel prior algorithm to defog. This method can effectively improve the brightness and contrast of the defog image, but it is prone to problems such as image distortion and noise amplification. As shown in Figure 8, Wang et al. (Reference Wang, Dong, Zhao, Houde and Ji2015) used infrared equipment to develop different target detection modes and detection algorithms for common maritime environments, heavy fog environments and backlit environments. Experiments show that these algorithms can accurately detect various sea surface targets under complex sea conditions.

In addition, most satellites also carry infrared lenses for telemetry to aid ship navigation. In 2003, Ahn et al. (Reference Ahn, Sohn and Hwang2003) used GMS-5 infrared data for comparison with clear sky radiation composite maps to detect sea fog with visibility less than 1 km . As shown in Figure 9, in 2014, Wu et al. (Reference Wu, Lu, Zhang and Yan2015) proposed a method to use CALIOP data to detect sea fog and improve the accuracy of MODIS sea fog detection. This method based on CALIOP data and MODIS data detects sea fog by comparing the differences between the two satellites, and uses an improved thresholding method to improve the accuracy of MODIS sea fog detection. In 2020, Ryu and Hong (Reference Ryu and Hong2020) used the near-infrared data of the Himawari-8 satellite to detect and forecast sea fog based on the NDSI algorithm, which improved the efficiency of sea fog detection under twilight conditions in the Yellow Sea region. However, this type of method is difficult to achieve higher time resolution detection.

Figure 9.

Infrared maritime target detection

(3) Maritime Intelligent Image Enhancement Algorithm

The physical model-based method is the earliest method used to defog marine fog images. This type of method establishes a physical model of fog, derives the fog concentration and fog depth in the image, and then performs defogging based on the model. For example, Li et al. (Reference Li, Wang, Peng and Peng2022) proposed a dehazing method based on an improved atmospheric scattering model, which can effectively remove haze in sea fog images. As shown in Figure 10, Van Nguyen et al. (Reference Van Nguyen, Mai and Lee2021) decompose foggy sky images into luminous and non-luminous layers using prior knowledge of texture and structure. A dehazing algorithm is then used to dehaze the non-luminous layer, and illumination compensation is used to restore the natural illumination in the luminous layer, resulting in a dehazed image. However, the physical model method requires high accuracy of input parameters and may have certain limitations in complex sea fog environments. The image enhancement method performs statistical analysis on the pixels in the sea fog image, which mainly includes extracting fog features using spatial domain, frequency domain, dark channel prior and other methods, and performing defogging based on the statistical results. For example, as shown in Figure 11, Wang et al. proposed a defogging method based on statistical analysis, which can effectively restore the clarity of sea fog images.

Figure 10.

Infrared target monitoring process

Figure 11.

Remote sensing sea fog detection

Ma et al. (Reference Ma, Wen and Liang2013) used dark channel prior knowledge to initially estimate the transmittance of sea images, and used guided image filtering to refine the initially estimated transmittance to dehazed sea fog images. Zhang et al. (Reference Zhang, Yang, Chen and Technologies2018) proposed a dehazing method based on histogram equalisation and wavelet transform, which can effectively improve the clarity of sea fog images. However, the feature analysis method requires high accuracy in fog feature extraction and statistical analysis, and there are many challenges in complex sea fog environments. Deep learning is an emerging method that has been applied to sea fog image denoising in recent years. This type of method builds a deep neural network model to learn the mapping between sea fog images and clear images, and performs dehaze based on the learning results. For example, Ma et al. (Reference Ma, Wen and Liang2013) developed a vision-based image dehazing method, which greatly improved the accuracy of obstacle detection and target tracking. Ren et al. (Reference Ren, Pan, Zhang and Cao2020) proposed a convolutional neural network-based dehazing method that can effectively restore the clarity of sea fog images. However, deep learning methods require a large amount of training data and computational resources, and there may be challenges in the selection of network structure as well as parameters.

3.2.3 Assisted navigation

Because there is no unified means for sensor fusion and decision making, the information fragmentation of AIS and other sensor devices in the cockpit is severe, which seriously affects the crew understanding the ship's status, resulting in accidents.

Assisted Vessel Navigation mainly relies on satellite navigation, radar, AIS and other technologies to automatically guide the vessel along the pre-set route, while providing information such as the vessel's position, course and speed to help the captain assess the vessel's position and improve vessel safety in foggy navigation. Therefore, situation awareness data for ship navigation often has multiple data sources. To fully exploit the value of multi-source data and ensure the accuracy and completeness of situational information, it is necessary to fuse and process multi-source situational awareness data. Since the sensing capabilities of sensors are reduced in foggy conditions, radar, satellite navigation, AIS and shore-based IoT devices are currently mainly used for marine auxiliary navigation. Information fusion methods mainly include Bayesian network (Guerriero et al., Reference Guerriero, Willett, Coraluppi and Carthel2008), D-S evidence theory (Juan et al., Reference Juan, Bo-Yan and Ji-Zhou2015), fuzzy theory, grey theory and neural network (Guerriero et al., Reference Guerriero, Willett, Coraluppi and Carthel2008), as shown in Figure 12 and 13. Due to poor visibility in foggy weather, crew need to use pilotage equipment. However, the VTMS centre operators can only see the straight line of the ship, so there will be a blind area when the ship turns. As shown in Figure 14, Simsir et al. (Reference Simsir, Amasyalí, Bal, Çelebi and Ertugrul2014) used satellite navigation and AIS data to design a decision support system for navigation using ANNS. In the experiment in the Istanbul Strait, the system could issue collision warning information three minutes in advance.

Figure 12.

Defog effect in different scenes based on physical model

Figure 13.

Defog effect in different scenes based on statistical model

Figure 14.

Ship route forecasting

In 2022, Zhang (Reference Zhang2021) used IOT technology to conduct research on the cooperative navigation and positioning method of an unmanned ship based on the data transmission and communication mechanism to monitor and adjust the position and course of the ship in real time. According to the position difference generated by the ships under the IOT technology exchange cycle, the cooperative control state of the ships is self-assembled, and the coordinated navigation position information is observed to realise cooperative navigation and positioning. The average positioning error generated by this navigation positioning method is small, with a value of 0$\cdot$014 m. Wang et al. (Reference Wang, Fan, Li, Min, Zhang and Chen2023), aiming at the requirements of safe navigation of ships in foggy conditions, used deep learning to construct a ship navigation fog environment classification model to achieve better fog navigation perception effects. Sorial et al. (Reference Sorial, Mouawad, Simetti and Odone2019) used radar point clouds to locate obstacles in the reference system, derive a scene graph containing obstacles and their headings, and obtain path navigation planning.

Although navigation equipment in fog can plan routes for fixed and moving obstacles in the waterway, it is inevitable that errors will occur due to various technical and human factors. According to statistics, more than 70% of accidents in foggy weather are collision accidents, and collisions between ships often cause huge property losses (Xie et al., Reference Xie, Chu, Liu and Wu2016). Geng et al. (Reference Geng, Wang, Wang and Baochen2019) proposed a maritime autonomous surface ship motion planning method based on dynamic programming by using a greedy algorithm, a greedy algorithm considering future collision risks, and a dynamic programming algorithm. They solved the collision avoidance and speed optimisation problems when sailing with other ships by establishing a velocity obstacle model. As a result, many scientists have conducted research on this problem. Wright (Reference Wright2019) explored the use of machine learning and artificial intelligence technologies as tools to integrate multiple sensor devices with collision avoidance navigation systems. The complementary benefits of multiple sensors are used to reduce the impact of environmental conditions on sensing capabilities. Han et al. (Reference Han, Cho, Kim and Kim2020) combined radar, lidar and cameras to create a new hybrid sensor fusion framework. Target detection algorithms were applied to hybrid sensor platforms to estimate collision probability. Finally, they plan collision avoidance navigational behaviour and control ship motion by training this information according to the Collision Prevention International Regulations at Sea (COLREG). The identification of small moving targets at sea is an important issue in ship navigation, especially for MASS. New means must be introduced to compensate for the shortcomings of radar and AIS in detecting small moving targets at sea. Chen et al. (Reference Chen, Chen, Zhang, Cheng and Wu2020) developed a novel hybrid deep learning method by combining an improved generative adversarial network and a convolutional neural network-based detection method. Specifically, they generate sufficiently informative artificial samples of small ships for training and learning between the generator and the discriminator.