Radar Point Clouds Get a Reality Check: YOLOv3 Beats Rivals in Landmark Detection Study

Automotive radar has long been the unsung hero of driver assistance systems. It keeps working when cameras are blinded by glare and lidar is smothered by fog, snow, or heavy rain, and it can measure the relative speed of objects in a single scan thanks to the Doppler effect. Yet when it comes to the machine-learning revolution that has transformed camera and lidar perception, radar has lagged behind — partly because nobody had rigorously compared the competing detection algorithms on the same footing. A new study published in AI Perspectives has now done exactly that, running five fundamentally different object detection architectures head-to-head on a large, open real-world radar data set, and the results overturn some common assumptions about which approach should lead the way.

Unlike camera images, radar point clouds are extraordinarily sparse — a single frame may contain only a scattering of reflected points, each carrying range, azimuth angle, amplitude, and Doppler velocity information. To build up enough density for machine learning, the researchers accumulated measurements over 500-millisecond time windows across a 100 by 100 meter area in front of the test vehicle, which carried four conventional 77 GHz radar sensors in its front bumper. The sensors resolve range to 0.15 meters and azimuth angle to under 2 degrees, with a cycle time of 60 milliseconds.

The data set itself is a crucial piece of the puzzle. The authors systematically ruled out the available public alternatives: nuScenes radar output is too sparse for pure radar experiments, the Astyx data set has too few recordings and suffers from bounding box errors, the Oxford Radar RobotCar, MulRan, and RADIATE sets use rotating marine-style radar that lacks the Doppler information essential to automotive sensing, and CARRADA provides only 2D projections rather than full point clouds. Their own proprietary data set — recently announced for public release on Zenodo — covers five object classes, including three vulnerable road user categories such as pedestrians and cyclists plus two vehicle classes, alongside a background class. The data was split into roughly 64 percent training, 16 percent validation, and 20 percent test sequences, with splits drawn at the sequence level to prevent near-duplicate samples from leaking between sets.

Five architectures entered the ring. The first was a modular pipeline: a two-stage DBSCAN-based clustering algorithm filtered the point cloud by velocity and grouped remaining points into object instances, then handed each cluster to an ensemble of 21 long short-term memory (LSTM) classifiers arranged in a one-versus-one and one-versus-all scheme. The second flipped that logic: a PointNet++ semantic segmentation network first assigned a class label to every individual point, and class-sensitive clustering then formed object instances, with a separate clustering run for each of the five object classes. The third approach converted the point cloud into image-like grid maps — one for maximum amplitude, two for signed Doppler extremes — and fed them to a YOLOv3 one-stage object detector with a Darknet-53 backbone. The fourth skipped the pseudo-image step entirely, using PointPillars, a point-cloud-native detector that learns its own intermediate representation. The fifth combined the segmentation network with the LSTM pipeline, using PointNet++ predictions to improve the clustering filter.

The preprocessing details turned out to matter enormously, and the paper devotes unusual care to them. Doppler values in the training data were heavily skewed toward zero, so the authors applied a monotonic polynomial transformation to widen the distribution and make the feature easier for the network to interpret. Grid map cells with many radar points propagated their values into empty neighboring cells, effectively blurring sparse detections outward. Grid maps were randomly rotated in multiples of 30 degrees during training for augmentation. For the clustering approach, the minimum neighbor count for DBSCAN was made range-dependent, acknowledging that the radar’s constant angular resolution means points thin out dramatically at longer distances. Amplitudes, which vary wildly even on a single object, were excluded from clustering entirely.

When the scores came in, YOLOv3 took the crown with a mean average precision of 53.96 percent at the strict intersection-over-union threshold of 0.5, leading the field by roughly one percentage point, and extending its margin to two points at the more lenient 0.3 threshold — a gap driven largely by its strong performance on trucks, whose large spatial extents it preserved better than competitors. The combined PointNet++ plus DBSCAN plus LSTM pipeline followed closely at 52.90 percent mAP, and actually beat YOLOv3 on the log average miss rate and F1 object score at the stricter threshold. The pure clustering-plus-LSTM approach without segmentation assistance managed 49.20 percent, while the random forest variant of the same pipeline reached 43.29 percent.

The biggest surprise came from PointPillars. In the lidar world, point-cloud-native detectors are generally expected to outperform image-based approaches because they learn the pseudo-image representation end-to-end rather than relying on hand-crafted grid mapping. Here, the original PointPillars was by far the worst performer at 36.89 percent mAP, struggling especially with pedestrians. The authors attribute this failure to the extreme sparsity of automotive radar data. When they swapped the original backbone for the deeper Darknet-53 module borrowed from YOLOv3, performance jumped by about nine points to 45.82 percent — evidence that the end-to-end concept is sound, but that current point-cloud architectures are not yet ready to exploit radar’s sparse returns as effectively as image-based methods.

One of the most striking findings concerned pedestrians, the class every method found hardest, likely because of their small size and few reflecting points. The PointNet++ segmentation approach, despite its mediocre overall ranking, achieved the best pedestrian detection of all methods. In a diagnostic experiment treating all object classes as a single road-user category — which strips out classification errors and measures pure object-versus-background detection — PointNet++ scored 69.21 percent, a nearly 30-point leap over its real mAP, while other methods gained only 5 to 10 points. This suggests the class-sensitive clustering generates abundant object proposals that pay off handsomely when classification is removed, hinting that with modest improvements in class discrimination, segmentation-based radar detection could become far more competitive.

Speed matters as much as accuracy for real vehicles, and here the results were encouraging across the board. The standard PointPillars ran in 13 milliseconds, the random forest variant of the clustering pipeline in 12.1 milliseconds, and the LSTM ensemble in 20.5 milliseconds, while YOLOv3 needed 32 milliseconds. Every method stayed under the 60-millisecond sensor cycle time, meaning all five could process data in real time even without code optimization. The authors also reported that ablation experiments confirmed the value of their preprocessing tricks: removing cell propagation and Doppler skewing slowed YOLOv3’s convergence from 275,000 training iterations to as many as 400,000, and attempts to refine detector boxes with DBSCAN or PointNet++ post-processing consistently failed to help, apparently because the box detectors were already correctly capturing outlier points that the refinement stages discarded.

The study’s broader message is that the choice between modular and end-to-end architectures is a genuine trade-off rather than a settled question. YOLOv3’s grid mapping closely resembles the preprocessing used for static radar object detection, opening a path toward unified networks that handle moving and stationary objects together — something current systems handle separately by exploiting Doppler signatures. Meanwhile, the modular LSTM pipeline offers easier component-wise upgrades, and its clustering stage has already been shown to benefit dramatically from next-generation high-resolution radar sensors. The authors argue that as denser sensors with elevation and polarimetric capabilities reach production vehicles, point-based methods may gain the most, and that anchor-free architectures like DETR could further improve robustness to radar’s notorious density variations. For a field where every research team has been developing on proprietary data, this open, systematic comparison provides the first representative benchmark — and a clear map of where radar perception research should go next.

Subject of Research: A systematic comparison of five machine learning architectures for detecting and classifying moving road users in automotive radar point clouds using a large open real-world data set.

Article Title: Object detection for automotive radar point clouds – a comparison

Article References: Scheiner, N., Kraus, F., Appenrodt, N., Dickmann, J., & Sick, B. (2021). Object detection for automotive radar point clouds – a comparison. AI Perspectives, 3(1), Article 6. https://doi.org/10.1186/s42467-021-00012-z

Image Credits: AI Generated

DOI: 10.1186/s42467-021-00012-z

Keywords: automotive radar, object detection, point clouds, YOLOv3, PointPillars, PointNet++, LSTM, machine learning, autonomous driving, Doppler velocity, semantic segmentation, DBSCAN clustering