I work with 3D point cloud registration of Terrestrial Laser Scanners (TLS) using Open3d (Python-3 library). After the registration it returns to me 2 metrics: FITNESS and RMSE, which are inversily proportional in ideal cases (low RMSE and high FITNESS = good registration). Here is what I know:
Fitness varies between 0 and 1 (0% - 100%) and measures, according with the documentation, "the overlapping area (# of inlier correspondences / # of points in target cloud). Higher is better".
The problem: is it realy a metric? A perfect registration will not give me 100%, the maximum value depends of the geometry of the clouds (the dataset). Sometimes the maximum ranges in 70-80% (clouds from indoor scenes) othertimes in 30-40% (clouds from outdoor scenes). The distance that I pass is the voxel used to amostrate the clouds (or 2x that).
The RMSE can range from 0 to unlimited, but usually ranges between 0.05 m (very good) and 0.50 m (very bad). The documentation only says "measures the RMSE of all inlier correspondences. Lower is better". Apparently it measures the RMSE of the correspondences found inside a given distance, you have to pass a distance, two clouds and a transformation T[Rot(3x3)+t(3x1)] to the fuction, so it returns RMSE and fitness found between the two clouds.
What pisses me off: none of the metrics can be trust. I can't use fitness to know if a registration was good because it depends of the scene, the only way is if I previously know the maximum fitness that can be found. And I can't use RMSE too, because sometimes the clouds are totaly wrong but the RMSE is low (which means good, but only if the fitness was good too). To make things worse: even with high RMSE and low Fitness (in relation to the maximum possible fitness) the clouds can be approximately correct, because a low rotation or translation between the two clouds causes both metrics to give bad results.
Turns out that the only way to say if a registration was good is visualizing both the clouds after the transformation, which make impossible to automate the verification. Is there a way to use both metrics at the same time to increase confidence in the result? I thougth using RMSE/Fitness (less is good) or Fitness/RMSE (more is good). My empirical experience says that fitness is more importante than RMSE.
I thank you in advance.
与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…