Skip to content

Fix contour filtering in ArUco #24479

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Nov 9, 2023

Conversation

AleksandrPanov
Copy link
Contributor

@AleksandrPanov AleksandrPanov commented Nov 1, 2023

Fixes #24220
Merge with opencv/opencv_extra#1117

Prepared new filtering algorithm for ArUco. This algorithm fixes:

  • multiple detection of a single marker;
  • incorrect work with nested contours (issue #24220);
  • detection charuco with big markers (big relative to the board cell)

Also:

  • This PR increases the marker grouping area by setting default minMarkerDistanceRate = 0.125 vs minMarkerDistanceRate = 0.05 earlier (all old values should work correctly)
  • Now a few candidates from the same group can be added to the list of candidates if they are far from each other. The number of saved contours depends on minGroupDistance.
  • This PR disables the ability to detect nested markers (only the innermost markers will be detected).

Tested new filtering algorithm for ArUco in synthetic benchmark with marker length rate = 0.8

  • Detected markers up from 95% to 100%
  • Error statistics for the new and the old algorithm (error in 10 pixels corresponds to not detected markers):

image

ArucoDetector apply several AdaptiveThreshold() with different params to improve contour detect in different conditions. This generates a series of contours that need to be filtered out:
image

The previous version of _filterTooCloseCandidates groups close contours and saves as candidate only one larger contour in group (or smaller contour in inverted marker case).

The new version of filterTooCloseCandidates groups close contours and saves as candidate one larger contour, and additionally saves several candidates located far enough away as "close candidates" (the average distance between any two saved markers greater than minGroupDistance*moduleSizeInPixels).

Also a hierarchy is also defined between the contours. First, the child contours are processed, then the parent ones. A contour containing an Aruco marker cannot be an Aruco marker and this contour is skipped.

aruco_h3

TODO: remove old filter fix in QR code detect in next PR

        // TODO: improve thinning Aruco markers
        bool isUniq = true;
        for (const FinderPatternInfo& tmp : patterns) {
            Point2f dist = pattern.center - tmp.center;
            if (max(abs(dist.x), abs(dist.y)) < 3.f * tmp.moduleSize) {
                isUniq = false;
                break;
            }
        }

Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

  • I agree to contribute to the project under Apache 2 License.
  • To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
  • The PR is proposed to the proper branch
  • There is a reference to the original bug report and related work
  • There is accuracy test, performance test and test data in opencv_extra repository, if applicable
    Patch to opencv_extra has the same branch name.
  • The feature is well documented and sample code can be built with the project CMake

@asmorkalov
Copy link
Contributor

test_aruco_dicts (test_charuco_board.aruco_objdetect_test) ... FAIL
test_aruco_marker_sizes (test_charuco_board.aruco_objdetect_test) ... FAIL

Testing OpenCV 4.8.0-dev
Local repo path: /home/ci/opencv
Discovering python tests from: /home/ci/build
    found 0 tests
======================================================================
FAIL: test_aruco_dicts (test_charuco_board.aruco_objdetect_test)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/ci/opencv/doc/pattern_tools/test_charuco_board.py", line 60, in test_aruco_dicts
    np.testing.assert_allclose(markerCorners_svg, markerCorners_cv, 0.1, 0.1)
  File "/usr/lib/python3/dist-packages/numpy/testing/_private/utils.py", line 1514, in assert_allclose
    assert_array_compare(compare, actual, desired, err_msg=str(err_msg),
  File "/usr/lib/python3/dist-packages/numpy/testing/_private/utils.py", line 841, in assert_array_compare
    raise AssertionError(msg)
AssertionError: 
Not equal to tolerance rtol=0.1, atol=0.1

Discovering python tests from: /home/ci/opencv/apps
    found 0 tests
Discovering python tests from: /home/ci/opencv/doc/pattern_tools
    found 2 tests
Mismatch: 71.4%
Max absolute difference: 3000.
Max relative difference: 30.
 x: array([[[[1100., 2100.],
         [1899., 2100.],
         [1899., 2899.],...
 y: array([[[[1100., 2100.],
         [1899., 2100.],
         [1899., 2899.],...

======================================================================
FAIL: test_aruco_marker_sizes (test_charuco_board.aruco_objdetect_test)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/ci/opencv/doc/pattern_tools/test_charuco_board.py", line 112, in test_aruco_marker_sizes
    np.testing.assert_allclose(markerCorners_svg, markerCorners_cv, 0.1, 0.1)
  File "/usr/lib/python3/dist-packages/numpy/testing/_private/utils.py", line 1514, in assert_allclose
    assert_array_compare(compare, actual, desired, err_msg=str(err_msg),
  File "/usr/lib/python3/dist-packages/numpy/testing/_private/utils.py", line 841, in assert_array_compare
    raise AssertionError(msg)
AssertionError: 
Not equal to tolerance rtol=0.1, atol=0.1

Mismatch: 71.4%
Max absolute difference: 3000.
Max relative difference: 8.
 x: array([[[[2375., 3375.],
         [2624., 3375.],
         [2624., 3624.],...
 y: array([[[[1375.,  375.],
         [1624.,  375.],
         [1624.,  624.],...

----------------------------------------------------------------------
Ran 2 tests in 2.509s

FAILED (failures=2)

Also there are build warnings.

@AleksandrPanov AleksandrPanov marked this pull request as ready for review November 7, 2023 23:04
Copy link
Contributor

@asmorkalov asmorkalov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

aruco.detectMarkers() fails at a certain angle while the image is perfect
2 participants