AI Image Tools May Help Autonomous Ships Drive Safely in the Arctic

Posted by PartYard Marine

Imagine an autonomous vessel navigating through one of the world’s most challenging oceanic regions. Sea ice is everywhere, and fog, snow, or rain severely limit visibility. Just as ship captains rely on their eyes, autonomous navigation systems depend on sensors, and bad weather can be just as challenging for sensors as it is for humans.

Overcoming Visibility Challenges

With the increase in Arctic shipping, technology that can eliminate the effects of bad weather from images, allowing algorithms to see as if it were a clear day, is invaluable. PhD candidate Nabil Panchi at NTNU’s Department of Marine Technology has developed an algorithm capable of this feat.

“We have introduced a new piece of the puzzle for improved sea ice modeling,” Panchi explained.

Existing AI algorithms perform well with clear images but struggle with blurry or weather-degraded visuals.

Panchi, also a naval architect, has trained his new algorithm using thousands of Arctic images to filter out visual obstructions like rain, snow, fog, and water droplets on camera lenses.

Panchi is part of the DigitalSeaIce project, which aims to integrate and digitalize Arctic sea ice observations and prediction models. The goal is to create a digital infrastructure combining regional sea ice forecasting models with local and satellite-based observations.

Deciphering the Arctic Through Imagery

“Our research focuses on understanding the Arctic environment via imagery. We are developing algorithms that function in all weather conditions,” Panchi stated.

His work is based on thousands of images taken during a voyage with the research ship Kronprins Haakon in the Arctic summer of 2023.

In partnership with his academic supervisor, Associate Professor Ekaterina Kim, Panchi recently published ‘Deep Learning Strategies for Analysis of Weather-Degraded Optical Sea Ice Images’ in the IEEE Sensors Journal.

Panchi and Kim propose two methods to help ships navigate safely in bad Arctic weather by “removing” weather effects from images. One approach uses artificial intelligence to clear up images, allowing existing algorithms to function correctly. Another, more efficient method involves creating new algorithms specifically designed for bad weather.

“Both approaches enable us to comprehend the Arctic in all weather conditions,” Nabil noted.

Urban Applications Meet Maritime Challenges

While algorithms that remove weather effects from images have long been used in urban settings for autonomous vehicles and surveillance, applying them to Arctic conditions is novel.

Current sea ice analysis algorithms rely on clear images taken from ships. However, Arctic images are often obscured by fog, rain, and snow, rendering them poor inputs for these algorithms.

These algorithms also need training to assess the type of ice around the ship, indicating safe areas to navigate or avoid.

First Public Sea Ice Image Dataset

To effectively remove fog and raindrops, algorithms must be trained on weather-affected sea ice images. “This research area has been largely overlooked due to limited access to clear Arctic images – until now. We hope our new open-access dataset spurs further development in weather-resilient technology,” Panchi said.

Under the guidance of Ekaterina Kim, Panchi has released the SeaIceWeather dataset online, marking the first publicly available collection of sea ice images.

Enhancing Navigation Safety

“There are very few open-access datasets like this. Creating them requires significant effort. We hope they see widespread use,” Panchi remarked.

Rainy conditions on one side and clear weather on the other. The AI model, when given a weather-affected image, removes the raindrops, presenting a clearer view of the ship’s surroundings.

Each image pair includes a ‘clean’ version with a clear view and a weather-obscured one. NTNU researchers aim for widespread use of the SeaIceWeather dataset, encouraging others to gather similar images.

Researchers in sea ice, navigation models, and dynamic positioning are primary users. These systems must operate in all weather conditions, and more image data improves monitoring, ice warnings, and navigation accuracy – a critical need.

An AI-based system for sea ice analysis aids crews in understanding their surroundings. “This information can develop advanced systems to prevent collisions, enhance navigation safety, and optimize sailing routes, ultimately reducing emissions,” said PhD candidate Nabil Panchi.

Growing Arctic Shipping and Inexperienced Captains

Global warming is causing sea ice to melt, increasing Arctic shipping. More companies are choosing these newly ice-free routes. Between 2013 and 2019, Arctic ship traffic rose by 25%.

“Navigating through sea ice requires significant experience. There are likely more ships in polar waters now than there are seasoned sea ice captains. Our system offers better support for ship operators,” Panchi noted.

Arctic sea ice is thinner, cracks easily, and forms large ice ridges or hummocks. From a ship’s bridge, only one meter of ice may be visible above the surface, hiding 4-5 meters below. The risk of hull damage is high, and not all ships are designed to break through ice.

An AI-based system for sea ice analysis assists crews in comprehending their surroundings. “This information helps develop advanced systems to avoid collisions, ensuring safer navigation and optimal routes, which also reduces emissions,” said PhD candidate Nabil Panchi.

Autonomous Shipping’s Future

Autonomous shipping has the potential to revolutionize the industry, making it more efficient and safer. According to Fortune Business Insights, the global autonomous ships market is projected to grow from $6.11 billion in 2024 to $12.25 billion by 2032.

“We anticipate more autonomous technology on ice-navigating ships, and current systems must be reliable in the extreme Arctic environment,” Kim stated.

30 Days of Image Collection

Panchi trained the algorithms on two image datasets: one from the GoNorth voyage on Kronprins Haakon, the other from online sources.

He mounted two cameras on one side of the ship, with one directly above the other. The upper camera had a clear view, while a transparent screen in front of the lower camera was sprayed with water to simulate raindrops.

From the observation room on the ninth deck, Panchi’s computer continuously downloaded images of sea ice. Over 30 days, he collected thousands of image pairs, each with one clear image and one with artificial rain.

Algorithm Training

The datasets include over 4600 clear images, primarily from the research voyage. Using algorithms, they created seven weather variants for each clear image: small, medium, and large snowflakes, rainy weather, fog, and real and simulated raindrops on the lens.

Based on these variants, they created two datasets: one indicating ice types around the ship and the other categorizing the ice (e.g., ice floes, pancake ice, ice slush, drift ice).

Three image-cleaning algorithms were trained on the datasets. Comparing results with clear images, researchers identified the most accurate algorithms for different weather conditions.

Daylight and Limited Weather Conditions

The method’s limitation is that all images were taken in broad daylight and included only three types of weather. Since the Arctic is dark from September to March, similar images should ideally be collected in winter. However, Augmented Reality (AR) can create artificial winter or night versions of existing images.

“Currently, researchers are the primary users of our work, but we hope it will be widely adopted in the future. Several factors influence when this will happen; it may take up to 5 years for the models to be commercially viable. They must be reliable assistants for ship management,” Nabil explained.

Emissions Reduction

Large ships consume enormous amounts of fuel, often sailing back and forth into the ice to break through, which is energy-intensive.

“Fully understanding the conditions around the vessel allows for efficient route planning, saving time, effort, and emissions. It also enhances shipping safety. More tankers carrying liquefied natural gas and other cargo are navigating the Arctic. No oil spills have occurred yet, but any incident would have severe consequences,” Panchi emphasized.

Untapped Image Data

Monitoring polar waters is crucial for understanding climate change. Many ships have cameras and sensors monitoring their course. While many ships produce images, few are available online. According to Panchi, most images end up in maritime data archives, rarely used except for a few insurance cases.

“We see significant potential in extracting useful data from these images. Our goal is to develop algorithms that can improve in real-time, on-site. Enhancing Arctic waters monitoring benefits society by providing a better basis for policy-making and ensuring sustainable and safe use of Arctic waters,” Panchi concluded.

Moving The Sea With Us!

Contact us today: +351 265 544 370 or go to Contacts Page


#PartYard #PartYardMarine #AutonomousShips #ArcticNavigation #SeaIce #AItechnology #MarineTechnology #ClimateChange #ArcticShipping #DigitalSeaIce #MachineLearning #ArtificialIntelligence #MaritimeSafety #OceanData #EnvironmentalTech #IceNavigation #PhDResearch #NTNU #TechInnovation #SustainableShipping #ArcticResearch #NavalArchitecture #ClimateTech #DataScience #MarineData #IceDetection #TechForGood #AutonomousNavigation #MovingTheSeaWithUs

Translate »