Real-time Object Detection From LiDAR Point Clouds

Project Overview

This project introduces an innovative solution leveraging Light Detection and Ranging (LiDAR) technology for real-time object detection during solar panel installation. The system is designed to enhance the automation of solar panel deployment in desert regions, addressing challenges of low manual efficiency and high operational costs.

Key Objectives

Real-time detection of solar rack poses
Determination of support structure positions and heights
Assessment of solar panel tilt angles
Integration with robotic installation systems

Watch Demo Video Download Report (PDF) View Source Code (GitHub)

Technical Framework

Our system implements a comprehensive pipeline for processing LiDAR data:

Data Acquisition and Preprocessing
- Reading geographical coordinates from YAML maps
- Subscribing to base link poses
- Coordinate transformation for support rods
- Point cloud fusion from dual LiDAR sensors
Perception Pipeline
- Statistical removal and PassThrough filtering for noise reduction
- Ground plane segmentation using Plane model
- Solar panel fitting and pose estimation
- Rod detection using Line model segmentation
- Inclination degree computation with safety alerts
Output and Visualization
- Custom message publishing at 10HzPortfolio
- Real-time visualization in RViz
- Integration with robot URDF
- Marker-based visualization of key points

Implementation Details

The system processes point cloud data through several key stages:

Point Cloud Processing
- Outlier removal and downsampling
- Ground plane removal
- Panel surface fitting
- Support rod detection
Parameter Extraction
- Ground plane parameters
- Panel center coordinates
- Roll, pitch, and yaw angles
- Rod top point coordinates
Safety Monitoring
- Real-time inclination monitoring
- Installation failure risk assessment
- Alert system for unsafe conditions