Collision-prone Track
Long-tail cases that test whether the ego can make progress while avoiding collisions.
Method
Agent-driven Simulation
Dataset
SemanticPlan Dataset
50+
scenario types
230+
closed-loop scenarios
Semantic Track
Evaluates whether ego behavior follows scenario semantics beyond collision avoidance.
General Semantic Scenarios
Scenario-specific constraints such as penalty regions, traffic-police instructions.
Honk-sensitive Scenarios
Cases where honking can help make progress, but can also be inappropriate around specific agents.
Results
Planning Results
Collision-prone Track
| Method | Prog. ↑ | Safe. ↑ | Overall ↑ |
|---|---|---|---|
| Rule-based | |||
| IDM | 0.818 | 0.507 | 0.409 |
| PDM Closed | 0.855 | 0.767 | 0.644 |
| Learning-based & Hybrid | |||
| UrbanDriver | 0.890 | 0.301 | 0.242 |
| GC-PGP | 0.604 | 0.432 | 0.258 |
| PlanTF | 0.861 | 0.459 | 0.380 |
| Diffusion Planner | 0.935 | 0.425 | 0.373 |
| PDM Hybrid | 0.856 | 0.774 | 0.651 |
| PLUTO | 0.924 | 0.664 | 0.616 |
Semantic Track
| Method | Gen. Sem. Prog. ↑ | Gen. Sem. Penalty ↓ | Honk Prog. ↑ | Honk Penalty ↓ | Overall ↑ |
|---|---|---|---|---|---|
| IDM | 0.580 | 0.574 | 0.499 | 0.000 | 0.353 |
| IDM + LLM | 0.600 | 0.561 | 0.562 | 0.024 | 0.389 |
| IDM + LLM (stop) | 0.395 | 0.146 | 0.335 | 0.004 | 0.309 |