ViP3D: End-to-end Visual Trajectory Prediction via 3D Agent Queries

ViP3D: End-to-end Visual Trajectory Prediction
via 3D Agent Queries

Junru Gu¹, Chenxu Hu¹, Tianyuan Zhang^2,3, Xuanyao Chen^2,4,
Yilun Wang⁵, Yue Wang⁶, Hang Zhao^1,2

¹ IIIS, Tsinghua University, ² Shanghai Qi Zhi Institute,
³ CMU, ⁴ Fudan University, ⁵ Li Auto, ⁶ MIT

* Equal contribution

Conference on Computer Vision and Pattern Recognition (CVPR), 2023

ViP3D is the first vision-based approach to predict future trajectories of agents for autonomous driving, modeling agent-level detection, tracking and prediction.

Github
Arxiv

Pipelines

The traditional pipeline involves multiple non-differentiable modules, e.g., detection, tracking, and prediction. ViP3D takes multi-view videos as input and generates predicted trajectories in an end-to-end manner, which can effectively leverage visual information such as turning signals of vehicles.

Related Projects

BEV Vectorized Mapping

VectorMapNet

BEV Detection

DETR3D

BEV Fusion

FUTR3D

BEV Tracking

MUTR3D

Citation

If you find our work intriguing, inspiring or useful to your research, please consider citing:

@article{vip3d,
title={ViP3D: End-to-end Visual Trajectory Prediction via 3D Agent Queries},
author={Gu, Junru and Hu, Chenxu and Zhang, Tianyuan and Chen, Xuanyao and Wang, Yilun and Wang, Yue and Zhao, Hang},
journal={arXiv preprint arXiv:2208.01582},
year={2022}
}