feat: add object detector feature in modmesh #619

rockleona · 2025-10-29T15:44:30Z

I started to have a research on computer vision, as the first step, this PR introduce Ultralytics YOLO as the object detection tool, and import yolov11 model as the base model.

Usage is really easy, just load an image, enable the detection, then you will see the result on the Console Widget. You can check the screenshot as below:

tigercosmos · 2025-10-29T15:51:56Z

cpp/modmesh/pilot/RVisionDockWidget.cpp

@@ -0,0 +1,162 @@
+/*
+ * Copyright (c) 2022, Yung-Yu Chen <[email protected]>


Change to your name if you are the first author.

tigercosmos · 2025-10-29T15:55:07Z

cpp/modmesh/pilot/RVisionDockWidget.hpp

+        QPushButton *toggleBtn = nullptr; 
+        bool enableDetection = false;
+    };
+    std::unique_ptr<Impl> m_impl;


I don't think we need to use pimpl idiom.

tigercosmos · 2025-10-29T15:56:30Z

cpp/modmesh/pilot/wrap_pilot.cpp

+
+    friend root_base_type;
+
+    WrapRVisionDockWidget(pybind11::module & mod, char const * pyname, char const * pydoc)


If there is nothing, maybe we don't need this?

tigercosmos · 2025-10-29T15:58:10Z

modmesh/pilot/_vision.py

+            logger.setLevel(logging.DEBUG)
+
+    if 'model' not in globals():
+        model = YOLO('./modmesh/pilot/yolo11n.pt')  # Please check the model path


How do you get the model? Maybe have a runtime download logic?

Yes, it will check the path if the model is exist during runtime, nor it will download it directly to the specified path.

I just found there is a directory called thirdparty, maybe I should specify the path overthere instead?

thirdparty is for the 3rd libraries. In this case, I think you can put the model file at the same directory of pilot runtime. Btw, it seems that the download logic is not implemented yet, right?

ultralytics library had already done the download logic, no need to do it agin, perhaps they will find the model name is in their file server or not, then download it when trigger class YOLO initialization.

tigercosmos · 2025-10-29T16:00:21Z

cpp/modmesh/pilot/RVisionDockWidget.cpp

+RVisionImage::RVisionImage(QWidget *parent)
+    : QWidget(parent)
+{
+    m_image.load(":/default.jpg");


You may want to put the image as an asset file.

If you add an image in the repository, make sure it is small and uses a compatible license.

Alternately, load an image from online.

tigercosmos · 2025-10-29T16:01:22Z

modmesh/pilot/_vision.py

@@ -0,0 +1,99 @@
+"""


Is it possible to write tests to validate the implementation?

How would you recommend to place the tests, put them in the tests/test_pilot.py perhaps?

I think you can put it at tests/test_vision.py?

yungyuc

Please make sure CI passes before requesting for review.

Correct copyright headers.
Remove unnecessary code like WrapRVisionDockWidget, which looks like a placeholder.
The pimpl class RVisionDockWidget::Impl is not necessary. Do not use pimpl.
Do not create a symbol named modmesh::BoundingBox.
Always add an end marker to classes and namespaces.

I see you are using pybind11 to call back into Python to use YOLO. Why don't you just write PySide6 to do it?

yungyuc · 2025-10-30T14:46:12Z

cpp/modmesh/pilot/RVisionDockWidget.hpp

+};
+
+class RVisionDockWidget;
+class Impl; 


The forward declaration Impl is no-op.

yungyuc · 2025-10-30T14:47:59Z

cpp/modmesh/pilot/RVisionDockWidget.hpp

+namespace modmesh
+{
+
+struct BoundingBox {


Do not name it as modmesh::BoundingBox, which will be used for other purposes.

Do not create additional namespace under modmesh. It will be over-engineering.

You may make it a nested class in RVisionDockWidget. A shorter name would make the code easier.

yungyuc · 2025-10-30T14:48:58Z

cpp/modmesh/pilot/RVisionDockWidget.hpp

+private:
+    QImage m_image;
+    std::vector<BoundingBox> m_boxes;
+};


Always keep an end marker:

... }; /* end class RVisionImage */

yungyuc · 2025-10-30T14:51:05Z

cpp/modmesh/pilot/RVisionDockWidget.cpp

+RVisionImage::RVisionImage(QWidget *parent)
+    : QWidget(parent)
+{
+    m_image.load(":/default.jpg");


If you add an image in the repository, make sure it is small and uses a compatible license.

Alternately, load an image from online.

yungyuc · 2025-10-30T14:51:44Z

cpp/modmesh/pilot/RVisionDockWidget.cpp

+    const uchar *data = rgbImg.bits();
+    py::array_t<uint8_t> np_img({height, width, channels}, data);
+    py::object vision_mod = py::module_::import("modmesh.pilot._vision");
+    py::object yolo_func = vision_mod.attr("yolo_detect");


If you call YOLO from Python, why not use PySide?

rockleona · 2025-11-17T03:47:21Z

Please make sure CI passes before requesting for review.

Correct copyright headers.

Remove unnecessary code like WrapRVisionDockWidget, which looks like a placeholder.

The pimpl class RVisionDockWidget::Impl is not necessary. Do not use pimpl.

Do not create a symbol named modmesh::BoundingBox.

Always add an end marker to classes and namespaces.

I see you are using pybind11 to call back into Python to use YOLO. Why don't you just write PySide6 to do it?

I thought it was a must to write all the GUI component with qt, I will change it to PySide6 since these functions were executed only from Python

yungyuc · 2025-11-17T11:31:45Z

@rockleona The code base has changed a lot. Please rebase to refresh the CI status.

tigercosmos reviewed Oct 29, 2025

View reviewed changes

yungyuc assigned rockleona Oct 30, 2025

yungyuc added the pilot GUI and visualization label Oct 30, 2025

yungyuc marked this pull request as draft October 30, 2025 14:42

yungyuc requested changes Oct 30, 2025

View reviewed changes

feat: add object detector feature in modmesh

79ad5a8

rockleona force-pushed the feat/yolov11 branch from 3a678c2 to 79ad5a8 Compare November 18, 2025 13:43

		@@ -0,0 +1,162 @@
		/*
		* Copyright (c) 2022, Yung-Yu Chen <[email protected]>


		friend root_base_type;

		WrapRVisionDockWidget(pybind11::module & mod, char const * pyname, char const * pydoc)

feat: add object detector feature in modmesh #619

Are you sure you want to change the base?

feat: add object detector feature in modmesh #619

Conversation

rockleona commented Oct 29, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rockleona Oct 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

yungyuc left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rockleona commented Nov 17, 2025

Uh oh!

yungyuc commented Nov 17, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

rockleona Oct 30, 2025 •

edited

Loading