Dual-Camera LiDAR Fusion for Occlusion-Robust 3D Detection in Urban Driving Simulation

Abstract

System overview: symmetric dual-camera LiDAR fusion combining PointPillar and CenterPoint 3D detection with YOLOv8 drone and forward-camera 2D detections for occlusion-robust perception.

Method

Overview of the proposed dual-camera LiDAR fusion pipeline. The PointPillar and CenterPoint 3D detectors and two independent YOLOv8 2D detectors produce detections from their respective sensors. The symmetric late-fusion module refines confidence scores based on uniform boost and suppress rules across both cameras.

Sensor configuration in CARLA Town10HD: ego vehicle with 64-channel LiDAR + forward camera, supplemented by a drone camera at 40m altitude.

Dual YOLOv8 detector architecture: independent models process drone (top-down) and forward camera (SDC) views.

Three-Stage Pipeline

• 3D LiDAR Detection (PointPillar + CenterPoint) — Processes ego vehicle’s 64-channel LiDAR point cloud into 3D bounding boxes with class labels and confidence scores using two complementary architectures
• 2D Camera Detection (YOLOv8) — Two independent YOLOv8m models process drone (top-down, 40m) and forward camera images, fine-tuned on CARLA-rendered data
• Symmetric Late Fusion — Camera-confirmed LiDAR detections receive confidence boosts (×1.15 single-cam, ×1.30 dual-cam), while unconfirmed low-confidence detections are suppressed (×0.75). Both cameras apply boost and suppress operations uniformly

Key Finding: Symmetric > Asymmetric

Counter-intuitively, applying both boost and suppress operations to both cameras outperforms asymmetric designs where the forward camera is restricted to boost-only. The forward camera’s value lies entirely in its ability to suppress false positives, not in boosting true positives.

Results

Evaluated on a CARLA Town10HD dataset with 2,600 frames, 35,837 annotations (Car + Pedestrian), and ten-seed repeated random sub-sampling validation.

PointPillar Results (10-seed average)

CenterPoint Results (10-seed average)

Ten-seed averaged mAP@0.5 with standard deviation error bars across PointPillar and CenterPoint detectors

Improvement over LiDAR-only baseline across all metrics and configurations. Bold borders indicate statistical significance.

Per-Seed Consistency

Per-seed mAP@0.5 consistency. All fusion configurations consistently outperform the LiDAR-only baseline across all ten random seeds (10/10 positive, sign test p = 0.001).

Camera Contribution Ablation (PointPillar)

The drone camera drives the majority of improvement, while the forward camera’s value lies in its ability to suppress false positives rather than boost true positives. Symmetric fusion outperforms asymmetric across both PointPillar and CenterPoint detectors, with all gains statistically significant (sign test p = 0.001, t-test p < 0.0001).

Sensitivity analysis of fusion hyperparameters (boost factor, suppress factor, confidence threshold). The optimal symmetric configuration achieves +0.92 pp mAP@0.5 improvement.

Qualitative Results

Representative scenarios showing LiDAR-only (left) vs dual-camera fused (right) detections. Fusion recovers occluded vehicles and suppresses false positives in cluttered scenes.

False positive suppression: the drone camera's overhead view confirms or denies LiDAR detections, reducing spurious boxes in occluded regions.

Multi-View Sample Pairs

Synchronized samples from all three sensors show how complementary viewpoints resolve ambiguities:

Drone camera view (40m altitude) — complete overhead coverage

Forward camera (SDC) view — frontal sector with depth perception

Side-by-side drone and SDC views for the same frame, showing complementary coverage areas.

Video Demonstrations

These videos show the fusion system operating in real-time on CARLA Town10HD sequences.

Fusion demonstration: combining LiDAR 3D detections with dual-camera 2D confirmations in real-time.

Citation

@article{zhou2026bevfusion,
  title={Dual-Camera LiDAR Fusion for Occlusion-Robust 3D Detection in Urban Driving Simulation},
  author={Zhou, Xingnan and Alecsandru, Ciprian},
  year={2026},
  note={In Preparation}
}