gsoc_2015: sfm module integration #290

edgarriba · 2015-06-30T13:17:01Z

opencv_extra: opencv/opencv_extra#279

Structure From Motion integration project

Migration of libmv DONE
Tests DONE
Existing documention DONE
Existing samples DONE
New samples (WIP)

vpisarev · 2015-07-01T09:20:12Z

@edgarriba, thanks for the PR. Putting libmv inside OpenCV is not a good idea. We will not be able to support such big chunk of code. Please, link to external libmv instead.
update: 👍

edgarriba · 2015-07-01T10:07:34Z

@vpisarev ok. I'll work on that. thks!

edgarriba · 2015-08-10T22:30:23Z

@vpisarev @vrabaud Is there any way to install libceres-dev or at least libeigen3-dev in the farm? The complete module depends on libmv_light which actually uses eigen matrices to compute everything.

alalek · 2015-08-11T08:34:51Z

There is no libceres-dev package in Ubuntu 14.04 default repo. We can build it from sources, but we need working instructions. Or install from PPA

edgarriba · 2015-08-11T08:49:02Z

Here it's well explained: http://ceres-solver.org/building.html#linux

besides, I'm on Ubuntu 14.04 and I have ceres in the repo

alalek · 2015-08-11T08:49:48Z

modules/sfm/CMakeLists.txt

+ocv_warnings_disable(CMAKE_CXX_FLAGS -Wundef -Winvalid-pch)
+
+# needed since we are linking static libs
+set(CMAKE_CXX_FLAGS ${CMAKE_CSS_FLAGS} "-fPIC")


This flag doesn't work with MSVS.
Please use something like this:
https://github.com/Itseez/opencv/blob/09b9b0fb9e9c9dd8c9e0d65705f8f19aa4c27f8a/3rdparty/zlib/CMakeLists.txt#L79-L83

alalek · 2015-08-11T09:41:28Z

@edgarriba Thanks. I will try.

root@linux-slave:/# apt-cache policy libceres-dev
N: Unable to locate package libceres-dev

It is docker-based Ubuntu image. No ROS features is added here by default.

alalek · 2015-08-11T11:11:52Z

Found Ceres version: 1.10.0 installed in: /usr/local

Installed on Linux slave.

Other builders (Mac,Win,android) are without Ceres, so sfm module should be disabled in these cases.

edgarriba · 2015-08-11T11:22:14Z

cool! thanks! what about your comment for module definition?

alalek · 2015-08-11T12:14:05Z

modules/sfm/CMakeLists.txt

+     set(CMAKE_CXX_FLAGS "${CMAKE_CSS_FLAGS} -fPIC")
+  endif()
+else()
+  ocv_module_disable(sfm)


UNIX condition is not right. It should be like this:

if (NOT HAVE_EIGEN) ocv_module_disable(sfm) endif()

edgarriba · 2015-08-11T14:09:51Z

@alalek why is core not found?

additionally, in local don't succeed compiing due to an Eigen error in core module:

/home/eriba/software/opencv/modules/core/include/opencv2/core/eigen.hpp:63:22: error: ‘Eigen’ does not name a type
void eigen2cv( const Eigen::Matrix<_Tp, _rows, _cols, _options, _maxRows, _maxCols>& src, Mat& dst )
^
/home/eriba/software/opencv/modules/core/include/opencv2/core/eigen.hpp:63:35: error: expected unqualified-id before ‘<’ token
void eigen2cv( const Eigen::Matrix<_Tp, _rows, _cols, _options, _maxRows, _maxCols>& src, Mat& dst )

Any idea?

alalek · 2015-08-11T14:32:01Z

It looks like missed #include or missed using namespace.

BTW, Buildbot shows this problem:

opencv_contrib/modules/sfm/src/projection.cpp:40:33: fatal error: opencv2/sfm/eigen.hpp: No such file or directory
 #include <opencv2/sfm/eigen.hpp>

Do you have this file? (Update: It was old build)

alalek · 2015-08-11T14:37:07Z

modules/sfm/CMakeLists.txt

+
+if (NOT HAVE_EIGEN)
+  ocv_module_disable(sfm)
+  message(STATUS "SfM module is disabled. Eigen library is required.")


ocv_module_disable is a macro with return(). This command is not executed.
You can place message before ocv_module_disable.

edgarriba · 2015-08-11T14:37:21Z

it's an old eigen cv2eigen implementation that was removed. Now I fixed the includes for this error (it's weird that my compiler didn't show me this error)

alalek · 2015-08-11T14:41:03Z

Try to add #include <Eigen/Core> before include of "opencv2/core/eigen.hpp"

alalek · 2015-08-11T14:46:21Z

modules/sfm/CMakeLists.txt

+
+ocv_set_module_sources( HEADERS ${OPENCV_SFM_HDRS} SOURCES ${OPENCV_SFM_SRC} )
+
+ocv_module_include_directories()


Place ocv_module_include_directories() on line 65 (before add_subdirectory(src/libmv_light))
This should resolve this problem:

opencv_contrib/modules/sfm/src/libmv_light/libmv/correspondence/feature.h:24:28: fatal error: opencv2/core.hpp: No such file or directory #include <opencv2/core.hpp>

edgarriba · 2015-08-11T15:03:42Z

working in local. Let's see

alalek · 2015-08-11T15:11:01Z

modules/sfm/CMakeLists.txt

+
+if(UNIX)
+  if(CMAKE_COMPILER_IS_GNUCXX OR CV_ICC)
+     set(CMAKE_CXX_FLAGS "${CMAKE_CSS_FLAGS} -fPIC")


I get this error:

t_triangulation.cpp.o
Building CXX object modules/sfm/CMakeFiles/opencv_test_sfm.dir/test/test_fundamental.cpp.o
[100%] Building CXX object modules/sfm/CMakeFiles/opencv_test_sfm.dir/test/scene.cpp.o
Linking CXX executable ../../bin/opencv_test_sfm
CMakeFiles/opencv_test_sfm.dir/test/test_triangulation.cpp.o: En la función checkTriangulation(int, int, bool, float, float) [clone .constprop.73]': test_triangulation.cpp:(.text._ZL18checkTriangulationiibff.constprop.73+0xce6): referencia acv::triangulatePoints(cv::_InputArray const&, cv::_InputArray const&, cv::_OutputArray const&)' sin definir
collect2: error: ld returned 1 exit status
make[2]: *** [bin/opencv_test_sfm] Error 1
make[1]: *** [modules/sfm/CMakeFiles/opencv_test_sfm.dir/all] Error 2
make[1]: *** Se espera a que terminen otras tareas....
Linking CXX shared library ../../lib/cv2.so
[100%] Built target opencv_python2
make: *** [all] Error 2

alalek · 2015-08-11T16:02:31Z

Currently I have this problem (build without ceres):

Linking CXX executable ../../bin/opencv_test_sfm
cd /home/alalek/projects/opencv_build/opencv_shared_debug/modules/sfm && /usr/bin/cmake -E cmake_link_script CMakeFiles/opencv_test_sfm.dir/link.txt --verbose=1
/usr/bin/c++    -fPIC -g  -O0 -DDEBUG -D_DEBUG       CMakeFiles/opencv_test_sfm.dir/test/test_precomp.cpp.o CMakeFiles/opencv_test_sfm.dir/test/test_projection.cpp.o CMakeFiles/opencv_test_sfm.dir/test/test_triangulation.cpp.o CMakeFiles/opencv_test_sfm.dir/test/test_reconstruct.cpp.o CMakeFiles/opencv_test_sfm.dir/test/test_robust.cpp.o CMakeFiles/opencv_test_sfm.dir/test/scene.cpp.o CMakeFiles/opencv_test_sfm.dir/test/test_main.cpp.o CMakeFiles/opencv_test_sfm.dir/test/test_simple_pipeline.cpp.o CMakeFiles/opencv_test_sfm.dir/test/test_conditioning.cpp.o CMakeFiles/opencv_test_sfm.dir/test/test_fundamental.cpp.o CMakeFiles/opencv_test_sfm.dir/test/test_numeric.cpp.o  -o ../../bin/opencv_test_sfm -rdynamic ../../lib/libopencv_ts.a ../../lib/libopencv_sfm.so.3.0.0 ../../lib/libopencv_imgcodecs.so.3.0.0 ../../lib/libopencv_videoio.so.3.0.0 ../../lib/libopencv_hal.a ../../lib/libopencv_core.so.3.0.0 ../../lib/libopencv_flann.so.3.0.0 ../../lib/libopencv_imgproc.so.3.0.0 ../../lib/libopencv_ml.so.3.0.0 ../../lib/libopencv_video.so.3.0.0 ../../lib/libopencv_imgcodecs.so.3.0.0 ../../lib/libopencv_shape.so.3.0.0 ../../lib/libopencv_videoio.so.3.0.0 ../../lib/libopencv_highgui.so.3.0.0 ../../lib/libopencv_features2d.so.3.0.0 ../../lib/libopencv_calib3d.so.3.0.0 ../../lib/libopencv_xfeatures2d.so.3.0.0 ../../lib/libopencv_hal.a ../../lib/libopencv_core.so.3.0.0 ../../lib/libopencv_imgproc.so.3.0.0 ../../lib/libopencv_imgcodecs.so.3.0.0 ../../lib/libopencv_videoio.so.3.0.0 ../../lib/libopencv_highgui.so.3.0.0 ../../lib/libopencv_hal.a ../../lib/libopencv_core.so.3.0.0 ../../lib/libopencv_flann.so.3.0.0 ../../lib/libopencv_imgproc.so.3.0.0 ../../lib/libopencv_ml.so.3.0.0 ../../lib/libopencv_video.so.3.0.0 ../../lib/libopencv_imgcodecs.so.3.0.0 ../../lib/libopencv_shape.so.3.0.0 ../../lib/libopencv_videoio.so.3.0.0 ../../lib/libopencv_highgui.so.3.0.0 ../../lib/libopencv_features2d.so.3.0.0 ../../lib/libopencv_calib3d.so.3.0.0 ../../lib/libopencv_xfeatures2d.so.3.0.0 -ldl -lm -lpthread -lrt /home/alalek/projects/opencv_dev/3rdparty/ippicv/unpack/ippicv_lnx/lib/intel64/libippicv.a ../../lib/libopencv_shape.so.3.0.0 ../../lib/libopencv_video.so.3.0.0 ../../lib/libopencv_calib3d.so.3.0.0 ../../lib/libopencv_features2d.so.3.0.0 ../../lib/libopencv_flann.so.3.0.0 ../../lib/libopencv_ml.so.3.0.0 ../../lib/libopencv_highgui.so.3.0.0 ../../lib/libopencv_videoio.so.3.0.0 ../../lib/libopencv_imgcodecs.so.3.0.0 ../../lib/libopencv_imgproc.so.3.0.0 ../../lib/libopencv_core.so.3.0.0 ../../lib/libopencv_hal.a -ldl -lm -lpthread -lrt /home/alalek/projects/opencv_dev/3rdparty/ippicv/unpack/ippicv_lnx/lib/intel64/libippicv.a -Wl,-rpath,/home/alalek/projects/opencv_build/opencv_shared_debug/lib 
/home/alalek/projects/opencv_build/opencv_shared_debug/modules/sfm/src/libmv_light/third_party/ssba/lib/libV3D.so: undefined reference to `ldl_ltsolve'
/home/alalek/projects/opencv_build/opencv_shared_debug/modules/sfm/src/libmv_light/third_party/ssba/lib/libV3D.so: undefined reference to `ldl_lsolve'
/home/alalek/projects/opencv_build/opencv_shared_debug/modules/sfm/src/libmv_light/third_party/ssba/lib/libV3D.so: undefined reference to `symamd_report'
/home/alalek/projects/opencv_build/opencv_shared_debug/modules/sfm/src/libmv_light/third_party/ssba/lib/libV3D.so: undefined reference to `ldl_perm'
/home/alalek/projects/opencv_build/opencv_shared_debug/modules/sfm/src/libmv_light/third_party/ssba/lib/libV3D.so: undefined reference to `symamd'
/home/alalek/projects/opencv_build/opencv_shared_debug/modules/sfm/src/libmv_light/third_party/ssba/lib/libV3D.so: undefined reference to `ldl_permt'
/home/alalek/projects/opencv_build/opencv_shared_debug/modules/sfm/src/libmv_light/third_party/ssba/lib/libV3D.so: undefined reference to `ldl_numeric'
/home/alalek/projects/opencv_build/opencv_shared_debug/modules/sfm/src/libmv_light/third_party/ssba/lib/libV3D.so: undefined reference to `ldl_symbolic'
/home/alalek/projects/opencv_build/opencv_shared_debug/modules/sfm/src/libmv_light/third_party/ssba/lib/libV3D.so: undefined reference to `ldl_dsolve'
collect2: error: ld returned 1 exit status
modules/sfm/CMakeFiles/opencv_test_sfm.dir/build.make:385: recipe for target 'bin/opencv_test_sfm' failed

Also I saw build failures with these messages:

no glog-devel package:

opencv_contrib/modules/sfm/src/libmv_light/libmv/logging/logging.h:24:26: fatal error: glog/logging.h: No such file or directory
 #include <glog/logging.h>

no gflags-devel package:

opencv_contrib/modules/sfm/src/libmv_light/libmv/multiview/euclidean_resection.cc:28:
/usr/include/glog/logging.h:85:27: fatal error: gflags/gflags.h: No such file or directory
 #include <gflags/gflags.h>

alalek · 2015-10-05T17:25:08Z

modules/sfm/CMakeLists.txt

+find_package(Ceres QUIET)
+
+if(NOT DEFINED SFM_DEPS_OK)
+  set(_fname "${CMAKE_CURRENT_BINARY_DIR}/test_sfm_deps.cpp")


Please add this code to resolve issue with ARM compiler (cross-compiler is buggy and has access to host include files in /usr/include , so we force check for glog library here):

if(NOT DEFINED GLOG_LIBRARIES) set(GLOG_LIBRARIES "glog") endif()

…o sfm

edgarriba · 2015-10-06T09:04:34Z

@alalek ARM compilation appears as not_queued

alalek · 2015-10-06T09:55:20Z

ARM build is not triggered automatically.
@edgarriba Thanks! Looks good to me
👍

edgarriba · 2015-10-06T10:17:51Z

@alalek what's ocv-carma2?

alalek · 2015-10-06T10:22:54Z

@edgarriba It is our second embedded ARM linux board with some network problems. This failure is expected at this moment. I check compilation step only, it is fine.

edgarriba · 2015-10-06T10:25:29Z

@alalek oki. Ping me if you want me to fix something

edgarriba · 2015-10-14T07:57:03Z

@alalek when it's supposed to be merged the PR?

…o sfm

alalek · 2015-10-14T14:25:18Z

Python binding knows and can handle only limited set of types, so there is no "std::string" support.
Also there are some issues with functions overloading (multiple functions with same base name, but different types for args).
Currently I don't know about working solutions for these problems.

edgarriba · 2015-10-22T08:02:12Z

@alalek just realized that header sfm.hpp it's not installed after "make install"

vpisarev · 2015-11-03T15:25:40Z

👍

edgarriba · 2015-11-03T15:46:00Z

@vpisarev there's an issue with installation

PhilLab · 2015-11-20T15:40:43Z

Awesome contribution! There seem to be two separate problems:

The test_sfm is only done under Linux (e.g. search for test_sfm here, you won't find it) - btw. there are several modules with no tests, what is the reason for this?
The buildbot says "Unable to read /build-2/precommit-contrib_linux64/opencv_extra/testdata/cv/sfm/backyard_tracks.txt" and I cannot find the file in the extra repository.

alalek · 2015-11-20T18:06:44Z

this module requires some pre-installed 3rdparty dependencies. They are installed only on Linux slave.
test data can be added to opencv_extra (test data should not exceed ~10 megabytes in general) with the same branch name (edgarriba/opencv_extra:sfm and buildbot will use it) or test can detect case of missing test data and "skip" processing with some message (message exists at this moment) instead of crash.

PhilLab · 2015-11-20T18:12:04Z

Ah, thanks for clarifying

edgarriba · 2015-11-20T18:17:39Z

@alalek right, need to commit test data.

- Previously, when Ceres was built as a static library we did not compile position independent code. This means that the resulting static library could not be linked against shared libraries, but could be used by executables. - To enable the use of a static Ceres library by other shared libraries as reported in [1], the static library must be generated from position independent code (except on Windows, where PIC does not apply). [1] opencv/opencv_contrib#290 (comment) Change-Id: I99388f1784ece688f91b162d009578c5c97ddaf6

vpisarev self-assigned this Jul 1, 2015

alalek reviewed Aug 11, 2015
View reviewed changes

edgarriba force-pushed the sfm branch from 1c105b6 to 2a14d22 Compare October 5, 2015 14:21

alalek reviewed Oct 5, 2015
View reviewed changes

edgarriba added 2 commits October 6, 2015 09:32

fixing ARM issue with glog

8fdf85e

Merge branch 'master' of https://github.com/Itseez/opencv_contrib int…

e496e9b

…o sfm

edgarriba added 2 commits October 14, 2015 10:40

Merge branch 'master' of https://github.com/Itseez/opencv_contrib int…

64204ea

…o sfm

adding python bindings

a1460ef

fixing building issue

9052277

edgarriba force-pushed the sfm branch from 0dc696c to 9052277 Compare October 14, 2015 16:33

opencv-pushbot merged commit 9052277 into opencv:master Dec 10, 2015

vpisarev added a commit that referenced this pull request Dec 10, 2015

Merge pull request #290 from edgarriba:sfm

5567c56

alalek mentioned this pull request Dec 29, 2015

Improve sparse pyrLK optical flow opencv/opencv#5771

Merged

alalek mentioned this pull request Jan 19, 2016

add io submodule for import scenes #509

Merged

alalek mentioned this pull request Oct 31, 2020

sfm: fix memory leak #2733

Merged


		ocv_set_module_sources( HEADERS ${OPENCV_SFM_HDRS} SOURCES ${OPENCV_SFM_SRC} )

		ocv_module_include_directories()

gsoc_2015: sfm module integration #290

gsoc_2015: sfm module integration #290

Conversation

edgarriba commented Jun 30, 2015

vpisarev commented Jul 1, 2015

edgarriba commented Jul 1, 2015

edgarriba commented Aug 10, 2015

alalek commented Aug 11, 2015

edgarriba commented Aug 11, 2015

Choose a reason for hiding this comment

Choose a reason for hiding this comment

alalek commented Aug 11, 2015

alalek commented Aug 11, 2015

edgarriba commented Aug 11, 2015

Choose a reason for hiding this comment

edgarriba commented Aug 11, 2015

alalek commented Aug 11, 2015

Choose a reason for hiding this comment

edgarriba commented Aug 11, 2015

alalek commented Aug 11, 2015

Choose a reason for hiding this comment

edgarriba commented Aug 11, 2015

Choose a reason for hiding this comment

Choose a reason for hiding this comment

alalek commented Aug 11, 2015

Choose a reason for hiding this comment

edgarriba commented Oct 6, 2015

alalek commented Oct 6, 2015

edgarriba commented Oct 6, 2015

alalek commented Oct 6, 2015

edgarriba commented Oct 6, 2015

edgarriba commented Oct 14, 2015

alalek commented Oct 14, 2015

edgarriba commented Oct 22, 2015

vpisarev commented Nov 3, 2015

edgarriba commented Nov 3, 2015

PhilLab commented Nov 20, 2015

alalek commented Nov 20, 2015

PhilLab commented Nov 20, 2015

edgarriba commented Nov 20, 2015