Switch keypoints to GstAnalytics metadata API by OskarFiedot · Pull Request #728 · open-edge-platform/dlstreamer

OskarFiedot · 2026-03-27T18:24:39Z

Description

Summary

Migrate keypoint handling from ad-hoc GstStructure tensors to the
GstAnalytics metadata API (GstAnalyticsGroupMtd, GstAnalyticsKeypointMtdV2),
and introduce a shared keypoint descriptor registry that maps semantic tags
to keypoint names and skeleton connections.

Changes

New files

gstanalyticskeypointdescriptor.h — shared C header defining
GstAnalyticsKeypointDescriptor structs with built-in descriptors for
COCO-17, OpenPose-18, and CenterFace-5 keypoint formats. Descriptors
store point names and skeleton connections as integer index pairs,
looked up by semantic tag at runtime.
gstanalyticsgroupmtd.{h,c} — GstAnalyticsGroupMtd implementation
for grouping keypoints with a semantic tag.
gstanalyticskeypointmtd.{h,c} — GstAnalyticsKeypointMtdV2
implementation for individual keypoint metadata.
gstvideometa_matrix.{h,c} — video meta matrix utilities.

tensor.h (convert_to_meta / convert_to_tensor)

convert_to_meta: reads skeleton from descriptor (zero-copy pointer)
instead of parsing string arrays. Removed GstAnalyticsClsMtd-based
point_names metadata — names are resolved from descriptors at read time.
convert_to_tensor: looks up point_names from descriptor via
semantic_tag. Skeleton reconstructed from RELATE_TO relations.
point_names and point_connections only emitted when non-empty.
Added Tensor::confidences() method that handles both scalar and
array-typed confidence fields.

Converters

yolo_v8.cpp, yolo_v26.cpp: replaced local name/connection string
vectors with COCO-17 descriptor reference. Skeleton connections are
now uint32_t index pairs.
centerface.cpp: uses CenterFace-5 descriptor. Emits per-keypoint
confidence as GST_TYPE_ARRAY. Fixed dims from {batch, 2*N} to
{N, 2}.
keypoints.h (base converter): sets structure name and type to
"keypoints" in createTensor().

gvawatermark

preparePrimsForKeypointConnections rewritten for uint32_t indices
via gst_value_array_* API. Removed get_keypoint_index_by_name.

gvametaconvert

attach_gvaluearray_to_json rewritten to use gst_value_array_* API
with support for string, uint, int, and float value types.
Confidence serialization: emits array only when values differ;
collapses to scalar when all identical.

gvametaaggregate

Handle unknown mtd types gracefully; removed unnecessary meta pointer
updates.

Validation

validateKeypoints skips per-keypoint confidence zeroing when all
values are identical (shared detection confidence).

Fixes # (issue)

Any Newly Introduced Dependencies

Please describe any newly introduced 3rd party dependencies in this change. List their name, license information and how they are used in the project.

How Has This Been Tested?

Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration

Checklist:

I agree to use the MIT license for my code changes.
I have not introduced any 3rd party components incompatible with MIT.
I have not included any company confidential information, trade secret, password or security token.
I have performed a self-review of my code.

github-actions

Remaining comments which cannot be posted as a review comment to avoid GitHub Rate Limit

pylint

github-actions · 2026-03-28T01:53:19Z

+    field = ET.Element("field", attrib={"name": name, "writable": "1"})
+    doc = ET.SubElement(field, "doc", attrib={"xml:space": "preserve"})
+    doc.text = doc_text
+    type_el = ET.SubElement(


[pylint] _{reported by reviewdog 🐶}
W0612: Unused variable 'type_el' (unused-variable)

github-actions · 2026-03-28T01:53:19Z

+    return True
+
+
+def main():


[pylint] _{reported by reviewdog 🐶}
C0116: Missing function or method docstring (missing-function-docstring)

github-actions · 2026-03-28T01:53:19Z

+        ET.indent(tree, space="  ", level=0)
+        tree.write(gir_path, xml_declaration=True, encoding="unicode")
+        # Add trailing newline
+        with open(gir_path, "a") as f:


[pylint] _{reported by reviewdog 🐶}
W1514: Using open without explicitly specifying an encoding (unspecified-encoding)

github-actions · 2026-03-28T01:53:19Z

@@ -0,0 +1,348 @@
+# ==============================================================================


[pylint] _{reported by reviewdog 🐶}
C0114: Missing module docstring (missing-module-docstring)

github-actions · 2026-03-28T01:53:19Z

+gi.require_version('GstAnalytics', '1.0')
+gi.require_version('DLStreamerMeta', '1.0')
+
+from gi.repository import Gst, GstAnalytics, DLStreamerMeta


[pylint] _{reported by reviewdog 🐶}
C0413: Import "from gi.repository import Gst, GstAnalytics, DLStreamerMeta" should be placed at the top of the module (wrong-import-position)

github-actions · 2026-03-28T01:53:19Z

+        self.assertFalse(grp.has_semantic_tag('test'))
+
+    def test_has_semantic_tag(self):
+        ok, grp = DLStreamerMeta.relation_meta_add_group_mtd(self.rmeta, 5)


[pylint] _{reported by reviewdog 🐶}
W0612: Unused variable 'ok' (unused-variable)

github-actions · 2026-03-28T01:53:19Z

+        self.assertTrue(grp.has_semantic_tag('hand-21-kp'))
+        self.assertFalse(grp.has_semantic_tag('pose-17-kp'))
+
+    def test_semantic_tag_has_prefix(self):


[pylint] _{reported by reviewdog 🐶}
C0116: Missing function or method docstring (missing-function-docstring)

github-actions · 2026-03-28T01:53:19Z

+        self.assertFalse(grp.has_semantic_tag('pose-17-kp'))
+
+    def test_semantic_tag_has_prefix(self):
+        ok, grp = DLStreamerMeta.relation_meta_add_group_mtd(self.rmeta, 5)


[pylint] _{reported by reviewdog 🐶}
W0612: Unused variable 'ok' (unused-variable)

github-actions · 2026-03-28T01:53:19Z

+        self.assertTrue(grp.semantic_tag_has_prefix('hand'))
+        self.assertFalse(grp.semantic_tag_has_prefix('pose'))
+
+    def test_add_member_and_get_member_count(self):


[pylint] _{reported by reviewdog 🐶}
C0116: Missing function or method docstring (missing-function-docstring)

github-actions · 2026-03-28T01:53:19Z

+        self.assertFalse(grp.semantic_tag_has_prefix('pose'))
+
+    def test_add_member_and_get_member_count(self):
+        ok, grp = DLStreamerMeta.relation_meta_add_group_mtd(self.rmeta, 5)


[pylint] _{reported by reviewdog 🐶}
W0612: Unused variable 'ok' (unused-variable)

github-actions · 2026-03-30T14:49:54Z

+        grp.add_member(kp2.id)
+        self.assertEqual(grp.get_member_count(), 2)
+
+    def test_get_member(self):


[pylint] _{reported by reviewdog 🐶}
C0116: Missing function or method docstring (missing-function-docstring)

github-actions · 2026-03-30T14:49:54Z

+        self.assertTrue(ok)
+        self.assertEqual(member.id, kp.id)
+
+    def test_iterate(self):


[pylint] _{reported by reviewdog 🐶}
C0116: Missing function or method docstring (missing-function-docstring)

github-actions · 2026-03-30T14:49:54Z

+
+        self.assertEqual(iterated_ids, kp_ids)
+
+    def test_get_mtd_type(self):


[pylint] _{reported by reviewdog 🐶}
C0116: Missing function or method docstring (missing-function-docstring)

github-actions · 2026-03-30T14:49:54Z

+        grp_type = DLStreamerMeta.group_mtd_get_mtd_type()
+        self.assertEqual(DLStreamerMeta.GroupMtd.get_mtd_type(), grp_type)
+
+    def test_relation_meta_get_group_mtd(self):


[pylint] _{reported by reviewdog 🐶}
C0116: Missing function or method docstring (missing-function-docstring)

github-actions · 2026-03-30T14:49:54Z

+        self.rmeta = GstAnalytics.buffer_add_analytics_relation_meta(
+            self.buffer)
+
+    def test_add_keypoints_group(self):


[pylint] _{reported by reviewdog 🐶}
C0116: Missing function or method docstring (missing-function-docstring)

github-actions · 2026-03-30T14:49:55Z

+        self.assertTrue(found, "GroupMtd not found during iteration")
+
+    def test_mixed_types_during_iteration(self):
+        ok, kp = DLStreamerMeta.relation_meta_add_keypoint_mtd(


[pylint] _{reported by reviewdog 🐶}
W0612: Unused variable 'kp' (unused-variable)

github-actions · 2026-03-30T14:49:55Z

+        ok, kp = DLStreamerMeta.relation_meta_add_keypoint_mtd(
+            self.rmeta, DLStreamerMeta.KeypointDimensions(2),
+            1, 2, 0, 0, 0.5)
+        ok, grp = DLStreamerMeta.relation_meta_add_group_mtd(self.rmeta, 5)


[pylint] _{reported by reviewdog 🐶}
W0612: Unused variable 'grp' (unused-variable)

github-actions · 2026-03-30T14:49:55Z

+        self.assertIn(DLStreamerMeta.KeypointMtd, types_found)
+        self.assertIn(DLStreamerMeta.GroupMtd, types_found)
+
+    def test_keypoints_group_iteration_types(self):


[pylint] _{reported by reviewdog 🐶}
C0116: Missing function or method docstring (missing-function-docstring)

github-actions · 2026-03-30T14:49:55Z

+    def test_keypoints_group_iteration_types(self):
+        positions = [10, 20, 30, 40]
+        confidences = [0.9, 0.8]
+        ok, grp = DLStreamerMeta.relation_meta_add_keypoints_group(


[pylint] _{reported by reviewdog 🐶}
W0612: Unused variable 'grp' (unused-variable)

github-actions · 2026-03-30T14:49:55Z

+
+# Import gstgva.region_of_interest to trigger _wrap_mtd() registration
+# which makes type() return proper DLStreamerMeta types during iteration
+from gstgva.region_of_interest import RegionOfInterest  # noqa: F401


[pylint] _{reported by reviewdog 🐶}
W0611: Unused RegionOfInterest imported from gstgva.region_of_interest (unused-import)

github-actions

Remaining comments which cannot be posted as a review comment to avoid GitHub Rate Limit

pylint

tests/unit_tests/tests_gstgva/test_keypoint_descriptor.py|132 col 4| C0116: Missing function or method docstring (missing-function-docstring)
tests/unit_tests/tests_gstgva/test_keypoint_descriptor.py|135 col 12| W0612: Unused variable 'from_idx' (unused-variable)
tests/unit_tests/tests_gstgva/test_keypoint_descriptor.py|135 col 22| W0612: Unused variable 'to_idx' (unused-variable)
tests/unit_tests/tests_gstgva/test_keypoint_descriptor.py|138 col 4| C0116: Missing function or method docstring (missing-function-docstring)
tests/unit_tests/tests_gstgva/test_keypoint_descriptor.py|148 col 4| C0116: Missing function or method docstring (missing-function-docstring)
tests/unit_tests/tests_gstgva/test_keypoint_descriptor.py|157 col 4| C0116: Missing function or method docstring (missing-function-docstring)
tests/unit_tests/tests_gstgva/test_keypoint_descriptor.py|20| R0904: Too many public methods (27/20) (too-many-public-methods)
tests/unit_tests/tests_gstgva/test_keypoint_descriptor.py|15| W0611: Unused GstAnalytics imported from gi.repository (unused-import)

github-actions · 2026-04-03T15:39:38Z

@@ -0,0 +1,168 @@
+# ==============================================================================


[pylint] _{reported by reviewdog 🐶}
C0114: Missing module docstring (missing-module-docstring)

github-actions · 2026-04-03T15:39:38Z

+gi.require_version('GstAnalytics', '1.0')
+gi.require_version('DLStreamerMeta', '1.0')
+
+from gi.repository import Gst, GstAnalytics, DLStreamerMeta


[pylint] _{reported by reviewdog 🐶}
C0413: Import "from gi.repository import Gst, GstAnalytics, DLStreamerMeta" should be placed at the top of the module (wrong-import-position)

github-actions · 2026-04-03T15:39:38Z

+gi.require_version('GstAnalytics', '1.0')
+gi.require_version('DLStreamerMeta', '1.0')
+
+from gi.repository import Gst, GstAnalytics, DLStreamerMeta


[pylint] _{reported by reviewdog 🐶}
E0611: No name 'Gst' in module 'gi.repository' (no-name-in-module)

github-actions · 2026-04-03T15:39:38Z

+gi.require_version('GstAnalytics', '1.0')
+gi.require_version('DLStreamerMeta', '1.0')
+
+from gi.repository import Gst, GstAnalytics, DLStreamerMeta


[pylint] _{reported by reviewdog 🐶}
E0611: No name 'GstAnalytics' in module 'gi.repository' (no-name-in-module)

github-actions · 2026-04-03T15:39:38Z

+gi.require_version('GstAnalytics', '1.0')
+gi.require_version('DLStreamerMeta', '1.0')
+
+from gi.repository import Gst, GstAnalytics, DLStreamerMeta


[pylint] _{reported by reviewdog 🐶}
E0611: No name 'DLStreamerMeta' in module 'gi.repository' (no-name-in-module)

github-actions · 2026-04-03T15:39:39Z

+            'face-landmarks/centerface-5')
+        self.assertEqual(desc.get_skeleton_connection_count(), 0)
+
+    def test_get_skeleton_connection_coco17_first(self):


[pylint] _{reported by reviewdog 🐶}
C0116: Missing function or method docstring (missing-function-docstring)

github-actions · 2026-04-03T15:39:39Z

+        self.assertEqual(from_idx, 0)  # nose
+        self.assertEqual(to_idx, 1)    # eye_l
+
+    def test_get_skeleton_connection_coco17_last(self):


[pylint] _{reported by reviewdog 🐶}
C0116: Missing function or method docstring (missing-function-docstring)

github-actions · 2026-04-03T15:39:39Z

+        self.assertEqual(from_idx, 14)  # knee_r
+        self.assertEqual(to_idx, 16)    # ankle_r
+
+    def test_get_skeleton_connection_out_of_range(self):


[pylint] _{reported by reviewdog 🐶}
C0116: Missing function or method docstring (missing-function-docstring)

github-actions · 2026-04-03T15:39:39Z

+
+    def test_get_skeleton_connection_out_of_range(self):
+        desc = DLStreamerMeta.KeypointDescriptor.lookup('body-pose/coco-17')
+        ok, from_idx, to_idx = desc.get_skeleton_connection(100)


[pylint] _{reported by reviewdog 🐶}
W0612: Unused variable 'from_idx' (unused-variable)

github-actions · 2026-04-03T15:39:39Z

+
+    def test_get_skeleton_connection_out_of_range(self):
+        desc = DLStreamerMeta.KeypointDescriptor.lookup('body-pose/coco-17')
+        ok, from_idx, to_idx = desc.get_skeleton_connection(100)


[pylint] _{reported by reviewdog 🐶}
W0612: Unused variable 'to_idx' (unused-variable)

github-actions · 2026-04-21T13:54:34Z

+        ok, from_idx, to_idx = desc.get_skeleton_connection(100)
+        self.assertFalse(ok)
+
+    def test_get_skeleton_connection_no_skeleton(self):


[pylint] _{reported by reviewdog 🐶}
C0116: Missing function or method docstring (missing-function-docstring)

github-actions · 2026-04-21T13:54:34Z

+    def test_get_skeleton_connection_no_skeleton(self):
+        desc = DLStreamerMeta.KeypointDescriptor.lookup(
+            'face-landmarks/centerface-5')
+        ok, from_idx, to_idx = desc.get_skeleton_connection(0)


[pylint] _{reported by reviewdog 🐶}
W0612: Unused variable 'from_idx' (unused-variable)

github-actions · 2026-04-21T13:54:35Z

+    def test_get_skeleton_connection_no_skeleton(self):
+        desc = DLStreamerMeta.KeypointDescriptor.lookup(
+            'face-landmarks/centerface-5')
+        ok, from_idx, to_idx = desc.get_skeleton_connection(0)


[pylint] _{reported by reviewdog 🐶}
W0612: Unused variable 'to_idx' (unused-variable)

github-actions · 2026-04-21T13:54:35Z

+        ok, from_idx, to_idx = desc.get_skeleton_connection(0)
+        self.assertFalse(ok)
+
+    def test_all_coco17_point_names(self):


[pylint] _{reported by reviewdog 🐶}
C0116: Missing function or method docstring (missing-function-docstring)

github-actions · 2026-04-21T13:54:35Z

+        names = [desc.get_point_name(i) for i in range(desc.get_point_count())]
+        self.assertEqual(names, expected)
+
+    def test_hrnet_coco17_shares_point_names_with_coco17(self):


[pylint] _{reported by reviewdog 🐶}
C0116: Missing function or method docstring (missing-function-docstring)

github-actions · 2026-04-21T13:54:35Z

+            self.assertEqual(coco.get_point_name(i),
+                             hrnet.get_point_name(i))
+
+    def test_hrnet_coco17_different_skeleton_from_coco17(self):


[pylint] _{reported by reviewdog 🐶}
C0116: Missing function or method docstring (missing-function-docstring)

github-actions · 2026-04-21T13:54:35Z

+Gst.init(sys.argv)
+
+
+class KeypointDescriptorTestCase(unittest.TestCase):


[pylint] _{reported by reviewdog 🐶}
R0904: Too many public methods (27/20) (too-many-public-methods)

github-actions · 2026-04-21T13:54:35Z

+gi.require_version('GstAnalytics', '1.0')
+gi.require_version('DLStreamerMeta', '1.0')
+
+from gi.repository import Gst, GstAnalytics, DLStreamerMeta


[pylint] _{reported by reviewdog 🐶}
W0611: Unused GstAnalytics imported from gi.repository (unused-import)

github-actions

Remaining comments which cannot be posted as a review comment to avoid GitHub Rate Limit

pylint

tests/unit_tests/tests_gstgva/test_tensor_convert.py|189 col 28| W0612: Unused variable 'dim' (unused-variable)
tests/unit_tests/tests_gstgva/test_tensor_convert.py|199 col 4| C0116: Missing function or method docstring (missing-function-docstring)
tests/unit_tests/tests_gstgva/test_tensor_convert.py|260 col 4| C0116: Missing function or method docstring (missing-function-docstring)
tests/unit_tests/tests_gstgva/test_tensor_convert.py|264 col 4| C0116: Missing function or method docstring (missing-function-docstring)
tests/unit_tests/tests_gstgva/test_tensor_convert.py|268 col 4| C0116: Missing function or method docstring (missing-function-docstring)
tests/unit_tests/tests_gstgva/test_tensor_convert.py|272 col 4| C0116: Missing function or method docstring (missing-function-docstring)
tests/unit_tests/tests_gstgva/test_tensor_convert.py|276 col 4| C0116: Missing function or method docstring (missing-function-docstring)
tests/unit_tests/tests_gstgva/test_tensor_convert.py|287 col 8| C0200: Consider using enumerate instead of iterating with range and len (consider-using-enumerate)
tests/unit_tests/tests_gstgva/test_tensor_convert.py|291 col 4| C0116: Missing function or method docstring (missing-function-docstring)
tests/unit_tests/tests_gstgva/test_tensor_convert.py|296 col 8| C0200: Consider using enumerate instead of iterating with range and len (consider-using-enumerate)
tests/unit_tests/tests_gstgva/test_tensor_convert.py|300 col 4| C0116: Missing function or method docstring (missing-function-docstring)
tests/unit_tests/tests_gstgva/test_tensor_convert.py|304 col 4| C0116: Missing function or method docstring (missing-function-docstring)
tests/unit_tests/tests_gstgva/test_tensor_convert.py|330 col 4| C0116: Missing function or method docstring (missing-function-docstring)
tests/unit_tests/tests_gstgva/test_tensor_convert.py|336 col 4| C0116: Missing function or method docstring (missing-function-docstring)
tests/unit_tests/tests_gstgva/test_tensor_convert.py|342 col 4| C0116: Missing function or method docstring (missing-function-docstring)
tests/unit_tests/tests_gstgva/test_tensor_convert.py|369 col 4| C0116: Missing function or method docstring (missing-function-docstring)
tests/unit_tests/tests_gstgva/test_tensor_convert.py|374 col 4| C0116: Missing function or method docstring (missing-function-docstring)
tests/unit_tests/tests_gstgva/test_tensor_convert.py|378 col 4| C0116: Missing function or method docstring (missing-function-docstring)
tests/unit_tests/tests_gstgva/test_tensor_convert.py|382 col 4| C0116: Missing function or method docstring (missing-function-docstring)
tests/unit_tests/tests_gstgva/test_tensor_convert.py|397 col 8| W0612: Unused variable 'ok' (unused-variable)
tests/unit_tests/tests_gstgva/test_tensor_convert.py|400 col 4| C0116: Missing function or method docstring (missing-function-docstring)
tests/unit_tests/tests_gstgva/test_tensor_convert.py|428 col 8| C0200: Consider using enumerate instead of iterating with range and len (consider-using-enumerate)
tests/unit_tests/tests_gstgva/test_tensor_convert.py|437 col 8| C0200: Consider using enumerate instead of iterating with range and len (consider-using-enumerate)
tests/unit_tests/tests_gstgva/test_tensor_convert.py|23| W0611: Unused RegionOfInterest imported from gstgva.region_of_interest (unused-import)

github-actions · 2026-04-23T14:01:08Z

+gi.require_version("DLStreamerMeta", "1.0")
+
+# pylint: disable=no-name-in-module
+from gi.repository import DLStreamerMeta


[pylint] _{reported by reviewdog 🐶}
C0413: Import "from gi.repository import DLStreamerMeta" should be placed at the top of the module (wrong-import-position)

github-actions · 2026-04-23T14:01:08Z

+
+# pylint: disable=no-name-in-module
+from gi.repository import DLStreamerMeta
+from gi.overrides import GstAnalytics as _GstAnalyticsOverride


[pylint] _{reported by reviewdog 🐶}
C0413: Import "from gi.overrides import GstAnalytics as _GstAnalyticsOverride" should be placed at the top of the module (wrong-import-position)

github-actions · 2026-04-23T14:01:08Z

-
-# Register Keypoint metadata
-# pylint: disable=protected-access
+from . import meta_registry  # noqa: F401 — registers DLStreamerMeta types with GstAnalytics


[pylint] _{reported by reviewdog 🐶}
E0402: Attempted relative import beyond top-level package (relative-beyond-top-level)

github-actions · 2026-04-23T14:01:08Z

-
-# Register Keypoint metadata
-# pylint: disable=protected-access
+from . import meta_registry  # noqa: F401 — registers DLStreamerMeta types with GstAnalytics


[pylint] _{reported by reviewdog 🐶}
W0611: Unused import meta_registry (unused-import)

github-actions · 2026-04-23T14:01:08Z

 from enum import Enum
 # pylint: disable=no-name-in-module
-from gi.repository import GObject, GstAnalytics, GLib
+from gi.repository import GObject, GstAnalytics, GLib, DLStreamerMeta


[pylint] _{reported by reviewdog 🐶}
C0413: Import "from gi.repository import GObject, GstAnalytics, GLib, DLStreamerMeta" should be placed at the top of the module (wrong-import-position)

github-actions · 2026-04-23T14:01:09Z

+from gstgva.region_of_interest import RegionOfInterest  # noqa: F401
+
+from gstgva.tensor import Tensor
+from gstgva.util import libgst


[pylint] _{reported by reviewdog 🐶}
C0413: Import "from gstgva.util import libgst" should be placed at the top of the module (wrong-import-position)

github-actions · 2026-04-23T14:01:09Z

+            GLib.quark_from_string("person"), OD_X, OD_Y, OD_W, OD_H, 0.9)
+        self.assertTrue(ok)
+
+    def test_returns_group_mtd(self):


[pylint] _{reported by reviewdog 🐶}
C0116: Missing function or method docstring (missing-function-docstring)

github-actions · 2026-04-23T14:01:09Z

+        self.assertIsNotNone(mtd)
+        self.assertIsInstance(mtd, DLStreamerMeta.GroupMtd)
+
+    def test_group_has_correct_member_count(self):


[pylint] _{reported by reviewdog 🐶}
C0116: Missing function or method docstring (missing-function-docstring)

github-actions · 2026-04-23T14:01:09Z

+        group = tensor.convert_to_meta(self.rmeta, self.od_mtd)
+        self.assertEqual(group.get_member_count(), KEYPOINT_COUNT)
+
+    def test_group_has_semantic_tag(self):


[pylint] _{reported by reviewdog 🐶}
C0116: Missing function or method docstring (missing-function-docstring)

github-actions · 2026-04-23T14:01:10Z

+            self.assertTrue(ok)
+            ok, kp = DLStreamerMeta.relation_meta_get_keypoint_mtd(self.rmeta, member.id)
+            self.assertTrue(ok)
+            ok, px, py, pz, dim = kp.get_position()


[pylint] _{reported by reviewdog 🐶}
W0612: Unused variable 'pz' (unused-variable)

github-actions · 2026-04-24T09:19:25Z

+            self.assertTrue(ok)
+            ok, kp = DLStreamerMeta.relation_meta_get_keypoint_mtd(self.rmeta, member.id)
+            self.assertTrue(ok)
+            ok, px, py, pz, dim = kp.get_position()


[pylint] _{reported by reviewdog 🐶}
W0612: Unused variable 'dim' (unused-variable)

github-actions · 2026-04-24T09:19:25Z

+            self.assertAlmostEqual(px, expected_x, delta=1, msg=f"keypoint {k} x mismatch")
+            self.assertAlmostEqual(py, expected_y, delta=1, msg=f"keypoint {k} y mismatch")
+
+    def test_keypoint_confidences(self):


[pylint] _{reported by reviewdog 🐶}
C0116: Missing function or method docstring (missing-function-docstring)

github-actions · 2026-04-24T09:19:25Z

+        self.assertIsNotNone(structure)
+        return Tensor(structure)
+
+    def test_name(self):


[pylint] _{reported by reviewdog 🐶}
C0116: Missing function or method docstring (missing-function-docstring)

github-actions · 2026-04-24T09:19:25Z

+        t = self._roundtrip_tensor()
+        self.assertEqual(t.name(), self.original.name())
+
+    def test_type(self):


[pylint] _{reported by reviewdog 🐶}
C0116: Missing function or method docstring (missing-function-docstring)

github-actions · 2026-04-24T09:19:25Z

+        t = self._roundtrip_tensor()
+        self.assertEqual(t.type(), self.original.type())
+
+    def test_format(self):


[pylint] _{reported by reviewdog 🐶}
C0116: Missing function or method docstring (missing-function-docstring)

github-actions · 2026-04-24T09:19:26Z

+    def setUp(self):
+        self.buffer = Gst.Buffer.new_allocate(None, 0, None)
+        self.rmeta = GstAnalytics.buffer_add_analytics_relation_meta(self.buffer)
+        ok, self.od_mtd = self.rmeta.add_od_mtd(


[pylint] _{reported by reviewdog 🐶}
W0612: Unused variable 'ok' (unused-variable)

github-actions · 2026-04-24T09:19:26Z

+        ok, self.od_mtd = self.rmeta.add_od_mtd(
+            GLib.quark_from_string("person"), OD_X, OD_Y, OD_W, OD_H, 0.9)
+
+    def test_full_roundtrip(self):


[pylint] _{reported by reviewdog 🐶}
C0116: Missing function or method docstring (missing-function-docstring)

github-actions · 2026-04-24T09:19:26Z

+        orig_data = original.data()
+        rest_data = restored.data()
+        self.assertEqual(len(orig_data), len(rest_data))
+        for i in range(len(orig_data)):


[pylint] _{reported by reviewdog 🐶}
C0200: Consider using enumerate instead of iterating with range and len (consider-using-enumerate)

github-actions · 2026-04-24T09:19:26Z

+        orig_conf = original.confidence()
+        rest_conf = restored.confidence()
+        self.assertEqual(len(orig_conf), len(rest_conf))
+        for i in range(len(orig_conf)):


[pylint] _{reported by reviewdog 🐶}
C0200: Consider using enumerate instead of iterating with range and len (consider-using-enumerate)

github-actions · 2026-04-24T09:19:26Z

+from gi.repository import Gst, GstAnalytics, GLib, DLStreamerMeta
+
+# Trigger _wrap_mtd() registration for DLStreamerMeta types
+from gstgva.region_of_interest import RegionOfInterest  # noqa: F401


[pylint] _{reported by reviewdog 🐶}
W0611: Unused RegionOfInterest imported from gstgva.region_of_interest (unused-import)

tjanczak · 2026-04-27T09:15:39Z

  - [Ubuntu advanced installation - build Docker image](./advanced_install/advanced_build_docker_image.md)
  - [Ubuntu advanced uninstall guide](./advanced_install/advanced_uninstall_guide.md)
 - [Metadata](./metadata.md)
+- [Keypoint Descriptor API](./keypoint_descriptor.md)


Please add keypoint_descriptor under metadata.md
We should update metadata.md to list supported GstAnalytics structures + list our extensions.

tjanczak · 2026-04-27T09:16:53Z

@@ -0,0 +1,90 @@
+/* GStreamer video meta matrix transform


Why do we need this file?

tjanczak · 2026-04-27T09:18:37Z

+    #  @param field_name name of the field
+    #  @param data list of values to store
+    #  @param value_type type hint: "float", "uint", or "string". If None, inferred from first element.
+    def set_vector(self, field_name: str, data: list, value_type: str = None) -> None:


Why do we need to extend tensory.py?
Can we use GstTensorMeta directly

tjanczak · 2026-04-27T09:21:13Z

 void attach_gvaluearray_to_json(const GVA::Tensor &tensor, const std::string &fieldname, json &jobject) {
-    GValueArray *valueArray = nullptr;
-    gst_structure_get_array(tensor.gst_structure(), fieldname.c_str(), &valueArray);
+    const GValue *garray = gst_structure_get_value(tensor.gst_structure(), fieldname.c_str());


again, should we just use GstAnalyticsTensor metadata directly

and remove unnecessary meta pointer update

…ons to integer indices Add a shared C header (gstanalyticskeypointdescriptor.h) that defines keypoint format descriptors — typed structs mapping a semantic_tag to keypoint names and skeleton connections as integer index pairs. Three built-in descriptors are provided: COCO-17, OpenPose-18, and CenterFace-5. Key changes: - Skeleton connections are now uint32_t index pairs instead of string name pairs throughout the pipeline (converters, watermark, metaconvert, tensor round-trip). - convert_to_meta (tensor.h): reads skeleton from the descriptor directly (zero-copy pointer) instead of parsing string arrays. Removed ClsMtd-based point_names metadata layer — names are now resolved from the descriptor at read time. - convert_to_tensor (tensor.h): looks up point_names from the descriptor via semantic_tag. Skeleton is still reconstructed from RELATE_TO relations. point_names and point_connections fields are only emitted when non-empty. - yolo_v8.cpp, yolo_v26.cpp: replaced local name/connection string vectors with a reference to the COCO-17 descriptor. - centerface.cpp: uses CenterFace-5 descriptor; emits per-keypoint confidence as GST_TYPE_ARRAY (float vector) instead of a single G_TYPE_DOUBLE, fixing downstream consumers that expect an array. - gvawatermark: preparePrimsForKeypointConnections rewritten to read uint32_t indices via gst_value_array_* API. Removed unused get_keypoint_index_by_name helper. - gvametaconvert: attach_gvaluearray_to_json rewritten to use gst_value_array_* API and handle multiple GValue types (string, uint, int, float). - keypoints.h (base converter): sets GstStructure name and type to "keypoints" in createTensor(); removed redundant format field. - blob_to_roi_converter.h, region_of_interest.h: filter keypoints tensors by type== instead of name==.

- Add Tensor::confidences() method that returns confidence as a vector of floats regardless of storage type (GST_TYPE_ARRAY or scalar G_TYPE_DOUBLE). Scalar values are returned as a single-element vector. - gvametaconvert: serialize confidence as a JSON array only when values differ; collapse to a single scalar when all values are identical. - validateKeypoints: skip per-keypoint confidence zeroing when all values are identical (shared detection confidence, e.g. CenterFace). - CenterFace: fix dims from {batch_size, 2*N} to {N, 2} so that watermark correctly draws all 5 facial landmarks instead of just one.

Remove V2 suffixes from GstAnalyticsKeypointMtd API to align with upstream GstAnalytics naming (KeypointMtdV2 -> KeypointMtd). Delete old gstanalyticskeypointsmtd.{h,c} (plural) containing the previous KeypointGroupMtd/KeypointSkeletonMtd types, now replaced by upstream-style GroupMtd from gstanalyticsgroupmtd.{h,c}. Add scripts/fix_gir_mtd_fields.py post-processing step to GIR generation pipeline, working around g-ir-scanner limitation where multiple typedefs aliasing the same struct tag produce records missing id/meta fields needed for Python caller-allocates bindings. Update all consumers: tensor.h, region_of_interest.h, metaaggregate.c, and Python gstgva bindings. Add 25 Python unit tests covering KeypointMtd, GroupMtd, keypoints_group convenience API, and _wrap_mtd type resolution.

gst_analytics_group_mtd_get_semantic_tag() now returns a newly allocated string (transfer full) instead of a borrowed internal pointer. This prevents use-after-free bugs when the underlying GstIdStr storage is modified or freed while the caller still holds the pointer. Changes: - Return g_strdup()'d copy from get_semantic_tag, caller must g_free() - Update all call sites in tensor.h and metaaggregate.c to g_free() the returned string after use - Update GIR annotations: transfer-ownership changed from "none" (nullable) to "full" - Minor code style fixes (include order, line wrapping)

…et_vector null safety Replace OpenPose 18 skeleton connections with the set matching human-pose-estimation-0001 model_proc (13 display edges derived from point_connections). Add a new HRNet COCO-17 keypoint descriptor (body-pose/hrnet-coco-17) with skeleton connections from single-human-pose-estimation-0001 model_proc, reusing the existing COCO-17 point names array. Apply the descriptors in keypoints_openpose and keypoints_hrnet converters so that each output tensor carries format, point_names and point_connections — the same metadata that yolo_v8/v26 converters already emit. Use the descriptor's point_count as the authoritative keypoints number in blob_to_meta_converter instead of parsing point_names from the model-proc file, removing the now-unused getKeypointsNumber helper. Guard Tensor::get_vector against null or non-array GValue fields to prevent GStreamer CRITICAL assertions when downstream elements (e.g. gvawatermark) query optional tensor fields like "confidence".

…ordinatesRestorer fallback Remove point_names and point_connections fields from model-proc JSON files for human-pose-estimation-0001 (OpenPose) and single-human-pose-estimation-0001 (HRNet). These fields are now provided by the keypoint descriptors added in the previous commit and no longer need to be duplicated in model-proc files. Update createCoordinatesRestorer() in converter_facade.cpp to detect keypoint converters by the "converter" field name when "point_names" is absent from model-proc. Without this fallback, removing point_names caused KeypointsCoordinatesRestorer to not be created, resulting in incorrect (unrestored) coordinate values in the output. Affected model-proc files: - gst_launch/human_pose_estimation/model_proc/human-pose-estimation-0001.json - model_proc/intel/human-pose-estimation-0001.json - model_proc/public/single-human-pose-estimation-0001.json

Exclude third-party files originating from upstream GStreamer that retain their original LGPL copyright headers (Collabora, Wim Taymans). These files cannot carry the Intel MIT header as that would misrepresent their license. Skipped files: - include/dlstreamer/gst/metadata/gstanalyticsgroupmtd.h - include/dlstreamer/gst/metadata/gstanalyticskeypointmtd.h - include/dlstreamer/gst/metadata/gstvideometa_matrix.h - src/gst/metadata/gstanalyticsgroupmtd.c - src/gst/metadata/gstanalyticskeypointmtd.c - src/gst/metadata/gstvideometa_matrix.c

List 6 source files originating from upstream GStreamer that are included in the repository under LGPL-2.0-or-later license: - gstanalyticsgroupmtd.{h,c} (Copyright Collabora Ltd) - gstanalyticskeypointmtd.{h,c} (Copyright Collabora) - gstvideometa_matrix.{h,c} (Copyright Wim Taymans) These files retain their original LGPL headers and must not be relicensed to MIT.

…ndings Move keypoint descriptor data (point names, skeleton connections) and lookup logic from static inline helpers in the header into a proper exported C source file (gstanalyticskeypointdescriptor.c). The header now declares a clean public API with GST_ANALYTICS_META_API-annotated functions matching the style of gstanalyticskeypointmtd.h. New public API: - gst_analytics_keypoint_descriptor_lookup() - gst_analytics_keypoint_descriptor_get_semantic_tag() - gst_analytics_keypoint_descriptor_get_point_count() - gst_analytics_keypoint_descriptor_get_point_name() - gst_analytics_keypoint_descriptor_get_skeleton_connection_count() - gst_analytics_keypoint_descriptor_get_skeleton_connection() The functions are picked up by g-ir-scanner, so the DLStreamerMeta-1.0 GIR/typelib now includes KeypointDescriptor with full Python bindings (DLStreamerMeta.KeypointDescriptor.lookup(), .get_point_name(), etc.). All C++ consumers (yolo_v8, yolo_v26, keypoints_hrnet, keypoints_openpose, blob_to_meta_converter, tensor.h) are updated to use the new pointer-based lookup API instead of the removed static variables. Also included: - Unit tests for the Python bindings (test_keypoint_descriptor.py) - Developer documentation (docs/user-guide/dev_guide/keypoint_descriptor.md)

- Introduce meta_registry.py for centralized metadata management - Update region_of_interest.py and tensor.py for metadata handling - Enhance util.py with utility functions - Add tensor conversion tests (C++ and Python) - Include keypoint detection sample pipeline - Update CMakeLists.txt for new tests

Replace hardcoded converter name strings ("keypoints_openpose", "keypoints_hrnet", "keypoints_3d") with static getName() calls from corresponding converter classes to improve maintainability.

github-actions Bot reviewed Mar 28, 2026

View reviewed changes

github-actions Bot reviewed Mar 30, 2026

View reviewed changes

OskarFiedot force-pushed the analytics_keypoint_mtd branch 3 times, most recently from c052512 to 986c7d4 Compare April 3, 2026 10:48

github-actions Bot reviewed Apr 3, 2026

View reviewed changes

OskarFiedot force-pushed the analytics_keypoint_mtd branch from d10ccbe to cf8d5ad Compare April 21, 2026 13:52

github-actions Bot reviewed Apr 21, 2026

View reviewed changes

OskarFiedot force-pushed the analytics_keypoint_mtd branch from a845b05 to 26bfe2f Compare April 23, 2026 13:59

github-actions Bot reviewed Apr 23, 2026

View reviewed changes

OskarFiedot force-pushed the analytics_keypoint_mtd branch from 26bfe2f to 853c484 Compare April 24, 2026 09:18

github-actions Bot reviewed Apr 24, 2026

View reviewed changes

tjanczak previously approved these changes Apr 27, 2026

View reviewed changes

OskarFiedot dismissed tjanczak’s stale review via 1b6f76d April 27, 2026 11:54

OskarFiedot force-pushed the analytics_keypoint_mtd branch from 1b6f76d to 2f43ce3 Compare April 27, 2026 13:27

OskarFiedot added 15 commits April 27, 2026 16:33

Switch to GStreamer metadata API for keypoint groups

127cfab

metaaggregate: handle unknown mtd types

282566e

and remove unnecessary meta pointer update

Fix headers and code style

f19bbad

Fix building on Windows by including the header files with the full path

81be95e

Refactor: use getName() instead of raw strings in converter_facade

f4d184b

Replace hardcoded converter name strings ("keypoints_openpose", "keypoints_hrnet", "keypoints_3d") with static getName() calls from corresponding converter classes to improve maintainability.

fix code style, pylint and headers

27b15ba

OskarFiedot force-pushed the analytics_keypoint_mtd branch 2 times, most recently from 559b9fe to 27b15ba Compare April 27, 2026 21:13

		@@ -0,0 +1,348 @@
		# ==============================================================================


		self.assertEqual(iterated_ids, kp_ids)

		def test_get_mtd_type(self):

		@@ -0,0 +1,168 @@
		# ==============================================================================

		Gst.init(sys.argv)


		class KeypointDescriptorTestCase(unittest.TestCase):

Conversation

OskarFiedot commented Mar 27, 2026

Description

Summary

Changes

New files

tensor.h (convert_to_meta / convert_to_tensor)

Converters

gvawatermark

gvametaconvert

gvametaaggregate

Validation

Any Newly Introduced Dependencies

How Has This Been Tested?

Checklist:

Uh oh!

github-actions Bot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions Bot Mar 28, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot Mar 28, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot Mar 28, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot Mar 28, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot Mar 28, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot Mar 28, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot Mar 28, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot Mar 28, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot Mar 28, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot Mar 28, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot Mar 30, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot Mar 30, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot Mar 30, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot Mar 30, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot Mar 30, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot Mar 30, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot Mar 30, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot Mar 30, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot Mar 30, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot Mar 30, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot left a comment