VCAD: Vision-Centric Autonomous Driving

What is VCAD?

VCAD (Vision-Centric Autonomous Driving) is an open-source research effort that pushes the frontiers of camera-centered autonomous driving technology. Existing autonomous driving technology is heavily relient on LiDAR sensing, however recent advances in AI show the great promise of vision-centered autonomous driving technology. Here are several important reasons to work on VCAD:

Rich Visual Attributes. Visual information contains rich attributes of the driving scene, which is essential for holistic scene understanding and decision making.
Vision is Scalable. Compared to LiDARs, RADARs, or other perception sensors, cameras are way more mature and pervasive. Vision technology is scalable in terms of data collection, compression, transmission and model training.
Vision-centric sensor fusion. We believe sensor fusion is critical for the future of autonomous driving, so we are actively pushing for vision-centric multi-sensory fusion technologies.

Projects on VCAD

HDMapNet	BEV Mapping: HDMapNet is an online HD map learning framework, whose goal is replace pre-annotated HD semantic maps.
DETR3D	BEV Detection: DETR3D is a multi-camera 3D object detection framework that does NOT require dense depth prediction or post-processing.
FUTR3D	BEV Fusion: FUTR3D is a unified sensor fusion framework that works with ARBITRARY sensor combinations.
MUTR3D	BEV Tracking: MUTR3D is an end-to-end multi-camera 3D tracking framework that works with arbrtary camera rigs.
VectorMapNet	BEV Vectorized Mapping: VectorMapNet is an end-to-end map learning framework that generates vectorized HD map from onboard sensor data.

Contributors

Hang Zhao Tsinghua University	Yilun Wang Li Auto	Yue Wang MIT
Justin Solomon MIT	Vitor Guizilini Toyota Research Institute	Tianyuan Zhang CMU
Qi Li Tsinghua University	Xuanyao Chen Fudan University	Yicheng Liu