AI论文

近年来，从鸟瞰视图（BEV）表示的进步已经展示了在车辆3D感知方面非常出色的前景。然而，虽然这些方法在标准基准测试中都取得了令人印象深刻的结果，但它们在各种条件下的稳健性仍然缺乏充分评估。在这项研究中，我们提出了RoboBEV，一个广泛的基准集，旨在评估BEV算法的稳健性。该集包括一个多样化的相机污染类型，每个类型都分别研究了3种严重程度。我们的基准还考虑了在使用多模态模型时发生的完整传感器故障的影响。通过RoboBEV，我们评估了包括检测、地图分割、深度估计和 occupancy prediction在内的33个最先进的BEV感知模型。我们的分析揭示了模型在分布数据上的性能与它在离散数据上的鲁棒性之间的显着相关。我们的实验结果还强调了预训练和深度无污染BEV转换策略在增强对离散数据鲁棒性的有效性。此外，我们还观察到，利用广泛的时序信息显著提高了模型的鲁棒性。根据我们的观察结果，我们基于CLIP模型设计了一种有效的鲁棒性增强策略。这个研究的结果为未来BEV模型的开发奠定了基础，这些模型将准确性和现实世界的鲁棒性无缝结合。

Recent advancements in bird’s eye view (BEV) representations have shown remarkable promise for in-vehicle 3D perception. However, while these methods have achieved impressive results on standard benchmarks, their robustness in varied conditions remains insufficiently assessed. In this study, we present RoboBEV, an extensive benchmark suite designed to evaluate the resilience of BEV algorithms. This suite incorporates a diverse set of camera corruption types, each examined over three severity levels. Our benchmarks also consider the impact of complete sensor failures that occur when using multi-modal models. Through RoboBEV, we assess 33 state-of-the-art BEV-based perception models spanning tasks like detection, map segmentation, depth estimation, and occupancy prediction. Our analyses reveal a noticeable correlation between the model’s performance on in-distribution datasets and its resilience to out-of-distribution challenges. Our experimental results also underline the efficacy of strategies like pre-training and depth-free BEV transformations in enhancing robustness against out-of-distribution data. Furthermore, we observe that leveraging extensive temporal information significantly improves the model’s robustness. Based on our observations, we design an effective robustness enhancement strategy based on the CLIP model. The insights from this study pave the way for the development of future BEV models that seamlessly combine accuracy with real-world robustness.

https://arxiv.org/abs/2405.17426

https://arxiv.org/pdf/2405.17426.pdf