Saving Marine Life with AI: A New Era in Ocean Conservation

In the battle against plastic pollution in our oceans, innovative technology has been the key to success. The Ocean Cleanup, a non-profit organisation dedicated to removing plastic debris from the ocean, teamed up with Deeper Insights to build an advanced AI system that enhances the detection and protection of marine life, particularly endangered sea turtles. The technology was complex, and we're excited to share some technical insights about this impactful solution.

‍

Why is AI needed in the first place?

The Ocean Cleanup's mission involves preventing new plastic waste from reaching the ocean via rivers and extracting existing debris from vast areas like the Great Pacific Garbage Patch. Their ocean-cleaning system, System 002, consists of an 800-metre floating barrier with a retention zone where plastic accumulates. However, this cleaning process inadvertently risks capturing marine life which requires careful monitoring and protection.

To protect marine life, the system relies on Protected Species Observers (PSOs), trained professionals responsible for monitoring the retention zone using a set of cameras installed on the system's camera skiff. The PSOs observe the camera feeds to detect the presence of turtles. The goal is to ensure that any sea life entering the retention zone is quickly detected so it can be safely released.

PSOs face high workload and fatigue as they are monitoring four camera feeds continuously for 12 hours can be physically and mentally exhausting, increasing the likelihood of errors. Moreover, the expertise of PSOs could be used in ways that are more beneficial for The Ocean Cleanup’s mission. Rather than spending long hours continuously monitoring camera feeds, their skills could be better employed in analysing data, improving monitoring protocols, and contributing to marine life conservation strategies.

To address these challenges, The Ocean Cleanup collaborated with our Deeper Insights’ AI and Computer Vision experts to develop a system that could identify animals in the retention zone with high precision, reducing the burden on PSOs. This collaboration has paved the way for more efficient and effective marine life protection in future operations.

Phase 1: Building an AI Solution

‍

Laying the Foundations

In the initial phase of the project, we focused on creating the preliminary infrastructure and foundational pipeline to build an AI-based turtle detection model. Our in-house Deeper Vision pipeline was adapted and customised to align with The Ocean Cleanup's data characteristics, providing seamless data ingestion, preprocessing, and inference.

To address image acquisition challenges, we implemented a comprehensive data processing pipeline using Normalised Mutual Information (NMI). This metric helped remove duplicate images caused by camera skiff-to-vessel connection issues. The result was an efficient dataset without sacrificing valuable information. We then worked on the dataset preparation methodology, ensuring optimal sampling and creating diverse training, validation, and test sets.

The selection and training of a suitable detection model was crucial. After evaluating several state-of-the-art models, we chose YOLOv8 for its great speed and accuracy characteristics. It was ideal for the project's real-time requirements. We adapted YOLOv8 to the project's unique pipeline and fine-tuned the hyperparameters for optimal performance. Once the model was adapted, we trained it using advanced augmentation techniques.

‍

Developing Enhanced Datasets

Since the existing sea life dataset was relatively small, we employed advanced data augmentation techniques to expand it. Leveraging the inpainting capabilities of Stable Diffusion XL, we artificially generated and augmented images containing turtles in varied contexts. To mimic real-world scenarios, shadows were added, and random objects like fish, plastic debris, and other objects were inserted into the images.

Once the model was trained and validated, we focused on deployment. A retraining script which allowed for model updates, and a Docker container with an API was created to provide real-time inference, ensuring seamless access to the trained model.

By the end of Phase 1, we had a robust detection model tailored to The Ocean Cleanup's specific needs, setting the stage for further refinement and optimisation in Phase 2.

‍

Phase 2: Refinement and Optimisation

Building on Phase 1's groundwork, Phase 2 focused on reducing false positives while maintaining high recall for detecting sea life. We began with a thorough analysis of the Phase 1 model's performance, identifying key areas for improvement.

False positives were predominantly triggered by fish, plastic debris, and image artefacts like shadows and blurs. Additionally, lower image quality due to changes in acquisition hardware presented new challenges for accurate detection. Recognizing these issues, we set out to refine the model's detection capabilities while maintaining efficiency.

‍

Verification with LLMs

Besides retraining the model with newly acquired data, post-processing methods were explored to enhance the model's accuracy. Different approaches were investigated, including zero/few-shot models like OWL-ViT and Grounding DINO. Although these models showed limited success, GPT-4 proved effective as a verification step for uncertain detections. When the YOLOv8 model's confidence fell below a certain threshold, GPT-4 was invoked to verify the presence of marine life based on textual prompts. By incorporating GPT-4 into the detection pipeline, we created a dual-layered verification system that significantly reduced false positives while ensuring cost-efficiency and real time inference.

‍

Phase 3: Future Possibilities and Advanced Technologies

The solution effectively performed its intended functions, yet like all technology, the initial version presents opportunities for refinement. Future phases could delve deeper into advanced 3D imaging and improved detection models to further reduce false positives and enhance recall.

One possibility includes deploying a sophisticated 3D camera setup for precise animal localisation and volumetric analysis of marine life and debris, providing a clearer view for more accurate identification. Additionally, integrating multimodal models like GPT-4 and Gemini could refine decision-making by assessing temporal data from videos, improving context and classification accuracy.

Exploring these advancements could set a new benchmark in marine conservation technology, ensuring The Ocean Cleanup's efforts remain as efficient and effective as possible.

The Core Technologies Used in AI-Driven Ocean Cleanup

The importance of this tool is clear: it enables more effective ocean cleanup while minimising harm to wildlife. However, the foundation of the solution rests on a sophisticated combination of technologies:

Machine Learning Models

Central to our AI-driven ocean cleanup initiative are advanced machine learning models, such as the YOLO (You Only Look Once) object detection system. These models are adept at analysing specific datasets to detect and categorise objects within vast data sets. Their applicability extends beyond marine debris identification to healthcare, where they help in the accurate diagnosis through medical imaging, and agriculture for the detection of pests and diseases affecting crops.

Real-Time Data Processing

The ability to process data in real-time is indispensable in environments that require immediate response based on the latest information. This technology supports dynamic decision-making in sectors like energy management, where it optimises the distribution based on current demand, and urban traffic management, facilitating the smooth flow of vehicles by adjusting traffic signals in real time.

Data Augmentation

Data augmentation techniques, such as inpainting, enrich training datasets, ensuring the robustness of AI models even when data is limited. This is crucial in sensitive fields such as healthcare, enhancing the training of AI systems without compromising patient privacy.

Computer Vision

Computer vision enables AI to interpret visual data, making it a key player in several industries. In retail, it can monitor customer movements and interactions to optimise store layouts and in manufacturing, it ensures quality control by spotting defects that are imperceptible to the human eye.

Predictive Analytics

Predictive analytics empower AI to forecast future scenarios based on historical data, which is pivotal for predictive maintenance in aviation and manufacturing. By predicting potential failures before they occur, businesses can save on costs and avert disasters, enhancing overall safety and efficiency.

Scalability

Finally, the scalability of AI solutions allows them to adapt to the varying demands of different industries. This flexibility ensures that businesses can scale their AI operations up or down based on real-time needs, optimising both performance and cost-effectiveness.

These core technologies not only pave the way for cleaner oceans but also spearhead advancements across various sectors, transforming both environmental strategies and business operations.

‍

How else could this technology be used?

The advantages of developing tools for specialised applications are significant; the core technology, insights, and methodologies can often be repurposed and adapted for use across a diverse range of industry sectors.

Agricultural Monitoring: Utilising AI for real-time monitoring and analysis of crop health, pest detection, and irrigation needs, optimising resource use and increasing yields.

Urban Traffic Management: Implementing AI to analyse traffic patterns, predict congestion, and optimise traffic light timings, improving flow and reducing urban congestion.

Manufacturing Process Optimisation: Employing AI to monitor production lines, predict equipment failures, and optimise manufacturing processes, reducing downtime and increasing efficiency.

Financial Fraud Detection: Leveraging AI to analyse transaction data for patterns of fraudulent activity, enhancing security measures and reducing losses in the financial sector.

Predictive Maintenance in Aviation: Implementing AI to monitor aircraft components and predict when maintenance is needed, enhancing safety and reducing unexpected downtime.

‍

Final Thoughts

Our collaboration with The Ocean Cleanup is a testament to the power of AI in tackling global challenges like ocean pollution. We have harnessed advanced technologies including YOLOv8 for marine life detection, GPT-4 for verification, and sophisticated data augmentation techniques to develop an innovative AI solution that significantly improves marine life protection during plastic collection.

This technology extends beyond ocean health, applying machine learning, real-time data processing, and computer vision to enhance sectors from healthcare to urban management. As AI continues to evolve, it opens up increasingly diverse opportunities to enhance our daily lives, businesses, and overall way of life.