Skip to content

feat(autoware_launch): add dbt camera cross validation#1757

Open
kaancolak wants to merge 3 commits intoautowarefoundation:mainfrom
kaancolak:feat/add-dbt-cross-camera-validation
Open

feat(autoware_launch): add dbt camera cross validation#1757
kaancolak wants to merge 3 commits intoautowarefoundation:mainfrom
kaancolak:feat/add-dbt-cross-camera-validation

Conversation

@kaancolak
Copy link
Contributor

Description

Add a separate roi_detected_object_fusion step for detection_by_tracker (DBT) objects in the detection pipeline. This filters phantom detections, particularly pedestrians that originate from the tracker feedback loop.

Problem

In the existing pipeline, the output of detection_by_tracker was fed directly into the merger/tracker without any camera validation.Once a momentary false-positive tracker was created for a static object misclassified as a pedestrian, detection_by_tracker would continuously find the point cloud cluster at that position and re-publish it as a pedestrian detection. Then detection_by_tracker would find the point cloud cluster at that position and re-publish it as a pedestrian detection. Since the DBT channel's can_spawn_new_tracker(link) is false but it can still update existing trackers with a default existence probability of 0.75, the tracker's total existence probability stays above the expiry threshold indefinitely, effectively living forever through the feedback mechanism.

detection_by_tracker_out.mp4

Solution

A roi_detected_object_fusion node is inserted in detection.launch.xml between the detection_by_tracker node output and all downstream consumers (both multi-channel tracker and merged-mode merger). The DBT node now outputs to an intermediate topic (detection_by_tracker/unfiltered/objects), which is validated against camera ROIs before being published to the original topic (detection_by_tracker/objects).

A dedicated parameter file (roi_detected_object_fusion_dbt.param.yaml) is used with stricter thresholds for PEDESTRIAN:

  • passthrough_lower_bound_probability_threshold: 1.0 → all DBT pedestrians must be confirmed by camera ROI (no passthrough)
  • trust_distance: 70.0m validation range
comparison_video_scene3.mp4

How was this PR tested?

Notes for reviewers

None.

Effects on system behavior

None.

Signed-off-by: Kaan Çolak <kaancolak95@gmail.com>
@github-actions
Copy link

github-actions bot commented Feb 16, 2026

Thank you for contributing to the Autoware project!

🚧 If your pull request is in progress, switch it to draft mode.

Please ensure:

@github-actions github-actions bot added the component:perception Advanced sensor data processing and environment understanding. (auto-assigned) label Feb 16, 2026
@kaancolak kaancolak self-assigned this Feb 16, 2026
@kaancolak kaancolak requested a review from mebasoglu February 16, 2026 19:29
@technolojin
Copy link
Contributor

Thank you for your work.

  • Idea: I understood and agree necessity of DbT filtering. DbT have issue of holding FP. We may have to check success cases, which DbT keep trackings when other detector (Centerpoint) has been failed.
  • Launcher: we have to make logic that "this DbT roi fusion filter shall not be used when perception mode is in [lidar, radar, lidar-radar]"

TOPIC share: Removing shape estimation and detection-by-tracker

I would like to remove shape estimation (incl. detection by tracker) from the perception pipeline in near future. the multi-object-tracker shall manage to properly utilize cluster objects (typically called 'unknown object'). Recent tracker updates (GIoU association, tracker motion model) leverage to directly use cluster objects.

This is for your information. This PR shall be reviewed without this topic.

kaancolak and others added 2 commits February 17, 2026 09:53
@kaancolak
Copy link
Contributor Author

@technolojin -san, Thanks for the reply and informations.

For point 2, I fixed it over topic manipulation: ROI fusion now has a dual guard (switch/detector/tracker_based AND switch/detector/camera_lidar) and the intermediate topic is only wired in camera modes, so in lidar, lidar_radar, and radar modes DBT outputs directly to detection_by_tracker/objects with no ROI fusion node launched.

@kaancolak
Copy link
Contributor Author

Friendly ping @YoshiRi @technolojin @MasatoSaeki @ogrex @yukkysaito

I’d appreciate your review on the PR when you’re available. Thank you.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

component:perception Advanced sensor data processing and environment understanding. (auto-assigned)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants