Structural Health Monitoring of Offshore Energy Deployments with Robotics and Deep Learning

The harsh ocean environment can structurally degrade ocean renewable energy installations from biofouling, fatigue cracking, spalling, and corrosion. Structural defects can lead to excessive downtime and increased operational and maintenance (O&M) expenses that can adversely impact the Levelized Cost of Energy (LCOE) from offshore wind and marine energy deployments. To ensure timely detection of structural defects and preventive maintenance interventions, effective Structural Health Monitoring (SHM) can significantly extend life and reduce the O&M costs and LCOE of ocean renewable energy systems. Traditional methods of SHM include fixed sensors that lack visual capability, and manual analysis of video recordings from Remotely Operated Vehicles (ROVs). The traditional inspection methods are time-consuming, expensive, and prone to human error. This project proposes an innovative SHM system that integrates a modified off-the-shelf ROV with a deep learning (DL) framework (YOLOv5) to process the recorded video data and process it to identify the defects. The deep learning framework is shown in Fig. 1. The video data collected from the ROV camera is stored in onboard memory and transferred to the DL model for processing at the completion of SHM campaign. The video recording is split into individual frames and passed through the preprocessing and augmentation stages for denoising, rotation, flipping, and Gaussian noise elimination. The Cross Stage Partial (CSP) layers are designed to split the input feature map, process each part separately, and merge them to improve efficiency. The spatial pyramid pooling layers (SPP) use multiple pooling scales to capture both global and local spatial information to detect objects of different sizes. The Feature Pyramid Network (FPN) layer enhances multi-scale feature representation and up-sample lower-resolution features and merges them with higher-resolution features to improve small object detection. The DL framework outputs include bounding box predictions, confidence scores and classification results. To collect the underwater images, the team deploys a Gladius Mini S underwater vehicle with an additional enclosure that contains data collection system (DCS). The DCS has onboard power and includes a 16 MP camera with temperature and depth sensors that communicate with and store data on a microcontroller. A depth sensor, an accelerometer, additional cameras, a sonar, and a gyroscope are also planned to be integrated into the system to add further capabilities. These sensors will extend the ROV’s capabilities of pathfinding, collision detection, positioning, and error correction, all of which contribute to making the autonomous operation of the vehicle possible. The images will be analyzed by the Convolutional Neural Network (CNN) to classify the defects. CNN is trained on annotated images stored in a defect database, and the preliminary results indicate significantly improved detection with 87% increase in accuracy compared to the traditional methods. By integrating the deep learning model with the capabilities of an underwater ROV, this project will increase the SHM time and cost efficiency of ocean renewable energy systems.