OpenCV Error: Assertion failed (type == srcB.type() && srcA.size() == srcB.size())

[Alberl-Lee]

OpenCV => 3.2.0
Operating System / Platform => ios 5/6/7
Compiler => Xcode 8.2.1

OpenCV Error: Assertion failed (type == srcB.type() && srcA.size() == srcB.size()) in mulSpectrums, file /Users/alberli/Documents/git/opencv_contrib_track/modules/core/src/dxt.cpp, line 3546

I used KCF, TLD, MIL, BOOSTING, MEDIANFLOW.
But the KCF and BOOSTING crash sometimes. Probably crash 3 times when run 5 times.

The code,

- (void)createKCF {
    trackerKCF = Tracker::create( "KCF" );
    bInit = true;
}

- (void)processImage:(cv::Mat&)image {
    // The KCF only support 3 channels, so i convert it to BGR.
    Mat bgr;
    cvtColor(image, bgr, CV_BGRA2BGR);
    image = bgr;

    // When i changed the select bound, it will init.
    if (bInit) {
        if(boundInit.height == 0 || image.rows==0 || image.cols==0)
            return;

        trackerKCF->init(image,boundInit);
        boundSelect = boundInit;
        bInit = false;
    }

    trackerKCF->update(image, boundSelect);
    rectangle( image, boundSelect, Scalar( 255, 0, 0 ), 2, 1 );
}

[alalek]

Could you provide stack trace for this assertion? (you can setup breakpoint on cv::error(...) function or on mentioned line in dxt.cpp)

[Alberl-Lee]

I am a beginer,not know how to debug the framework, and mainly the opencv/platforms/ios/ios/build/arm64-iPhoneOS/OpenCV.xcodeproj not generate the opencv2.framework, it geneate libopencv_tracking.a , libopencv_world.a, and some .a file.
(How can i generate the OpenCV.xcodeproj of the opencv2.framework ?)
so I can only provide the log:

2017-03-10 11:25:12.116354 KCF[2972:1412156] [MC] System group container for systemgroup.com.apple.configurationprofiles path is /private/var/containers/Shared/SystemGroup/systemgroup.com.apple.configurationprofiles
2017-03-10 11:25:12.118168 KCF[2972:1412156] [MC] Reading from public effective user settings.
2017-03-10 11:25:12.119485 KCF[2972:1412156] camera available: YES
2017-03-10 11:25:12.144180 KCF[2972:1412156] deviceOrientationDidChange: 0
2017-03-10 11:25:12.144268 KCF[2972:1412156] deviceOrientationDidChange: 0
2017-03-10 11:25:12.145145 KCF[2972:1412156] deviceOrientationDidChange: 1
2017-03-10 11:25:12.181699 KCF[2972:1412156] [Camera] device connected? YES
2017-03-10 11:25:12.181759 KCF[2972:1412156] [Camera] device position back
2017-03-10 11:25:12.183790 KCF[2972:1412156] [Camera] FPS set to 30
2017-03-10 11:25:12.184476 KCF[2972:1412156] layout preview layer
2017-03-10 11:25:12.184570 KCF[2972:1412156] [Camera] created AVCaptureVideoDataOutput

it crashed very frequent, you can recurrent easyly.

[Alberl-Lee]

@alalek Hi, i have learn how to debug the framework.
the KCF crash error is:
OpenCV Error: Assertion failed ((unsigned)(i1 * DataType<_Tp>::channels) < (unsigned)(size.p[1] * channels())) in at, file /Users/alberli/Documents/git/opencv_contrib_track/modules/core/include/opencv2/core/mat.inl.hpp, line 956

stack is:
#1 0x0000000100351ef4 in cv::error(cv::Exception const&) at /Users/alberli/Documents/git/opencv_contrib_track/modules/core/src/system.cpp:661
#2 0x0000000100351268 in cv::error(int, cv::String const&, char const*, char const*, int) at /Users/alberli/Documents/git/opencv_contrib_track/modules/core/src/system.cpp:666
#3 0x000000010012c5f0 in double& cv::Mat::at(int, int) at /Users/alberli/Documents/git/opencv_contrib_track/modules/core/include/opencv2/core/mat.inl.hpp:956
#4 0x000000010012b3ac in cv::TrackerKCFImpl::initImpl(cv::Mat const&, cv::Rect_ const&) at /Users/alberli/Documents/git/opencv_contrib_track/opencv_contrib/modules/tracking/src/trackerKCF.cpp:225
#5 0x0000000100119438 in cv::Tracker::init(cv::Mat const&, cv::Rect_ const&) at /Users/alberli/Documents/git/opencv_contrib_track/opencv_contrib/modules/tracking/src/tracker.cpp:76

Crash at function:

_Tp& Mat::at(int i0, int i1)
{
...
    CV_DbgAssert((unsigned)(i1 * DataType<_Tp>::channels) < (unsigned)(size.p[1] * channels()));
...
}

_Tp is double,
i0 = 0,
i1 = 142,
channels() = 1,
(unsigned)size.p[0] = 195,
(unsigned)size.p[1] = 142, size.p[1] = eeiis1.meeeeea

so, the CV_DbgAssert is 142 * 1 < 142 *1 , should fix < to <= ?

The function :

bool TrackerKCFImpl::initImpl( const Mat& /*image*/, const Rect2d& boundingBox ){
...
y.at<double>(i,j)=(i-roi.height/2+1)*(i-roi.height/2+1)+(j-roi.width/2+1)*(j-roi.width/2+1);
...
}

boundingBox const Rect2d &
x double 103.66665649414063
y double 139.33332824707031
width double 142.33334350585938
height double 195.66667175292969

roi cv::Rect2d
x double 16.249992370605469
y double 20.749996185302734
width double 142.33334350585938
height double 195.66667175292969

i unsigned int 0
j unsigned int 142
y cv::Mat :
'
flags int 1124024326
dims int 2
rows int 195
cols int 142
data uchar * 0x10379c010 0x000000010379c010
datastart const uchar * 0x10379c010 0x000000010379c010
dataend const uchar * "" 0x00000001037d2160
datalimit const uchar * "" 0x00000001037d2160
allocator cv::MatAllocator * NULL 0x0000000000000000
u cv::UMatData * 0x1702a2a60 0x00000001702a2a60
prevAllocator cv::StdMatAllocator * 0x170005140 0x0000000170005140
currAllocator cv::StdMatAllocator * 0x170005140 0x0000000170005140
urefcount int 0
refcount int 1
data uchar * 0x10379c010 0x000000010379c010
origdata uchar * 0x10379c010 0x000000010379c010
size size_t 221520
flags int 0
handle void * NULL 0x0000000000000000
userdata void * NULL 0x0000000000000000
allocatorFlags_ int 0
mapcount int 0
originalUMatData cv::UMatData * NULL 0x0000000000000000
size cv::MatSize
p int * 0x1348ca970 0x00000001348ca970
*p int 195
step cv::MatStep
p size_t * 0x1348ca9b8 0x00000001348ca9b8
*p size_t 1136
buf size_t [2]
[0] size_t 1136
[1] size_t 8
'

[Alberl-Lee]

The BOOSTING crash error is:
OpenCV Error: Assertion failed (data) in at, file /Users/alberli/Documents/git/opencv_contrib_track/modules/core/include/opencv2/core/mat.inl.hpp, line 1013

stack is:
#1 0x0000000100289ef4 in cv::error(cv::Exception const&) at /Users/alberli/Documents/git/opencv_contrib_track/modules/core/src/system.cpp:661
#2 0x0000000100289268 in cv::error(int, cv::String const&, char const*, char const*, int) at /Users/alberli/Documents/git/opencv_contrib_track/modules/core/src/system.cpp:666
#3 0x000000010002be58 in float const& cv::Mat::at(int) const at /Users/alberli/Documents/git/opencv_contrib_track/modules/core/include/opencv2/core/mat.inl.hpp:1013
#4 0x000000010002afd8 in cv::BaseClassifier::eval(cv::Mat const&) at /Users/alberli/Documents/git/opencv_contrib_track/opencv_contrib/modules/tracking/src/onlineBoosting.cpp:275
#5 0x000000010002af30 in cv::StrongClassifierDirectSelection::eval(cv::Mat const&) at /Users/alberli/Documents/git/opencv_contrib_track/opencv_contrib/modules/tracking/src/onlineBoosting.cpp:205
#6 0x0000000100028ed4 in cv::Detector::classifySmooth(std::__1::vector<cv::Mat, std::__1::allocatorcv::Mat > const&, float) at /Users/alberli/Documents/git/opencv_contrib_track/opencv_contrib/modules/tracking/src/onlineBoosting.cpp:611
#7 0x0000000100028c38 in cv::StrongClassifierDirectSelection::classifySmooth(std::__1::vector<cv::Mat, std::_1::allocatorcv::Mat > const&, cv::Rect const&, int&) at /Users/alberli/Documents/git/opencv_contrib_track/opencv_contrib/modules/tracking/src/onlineBoosting.cpp:104
#8 0x000000010009a870 in cv::TrackerStateEstimatorAdaBoosting::estimateImpl(std::__1::vector<std::__1::vector<std::__1::pair<cv::Ptrcv::TrackerTargetState, float>, std::__1::allocator<std::__1::pair<cv::Ptrcv::TrackerTargetState, float> > >, std::__1::allocator<std::__1::vector<std::__1::pair<cv::Ptrcv::TrackerTargetState, float>, std::__1::allocator<std::__1::pair<cv::Ptrcv::TrackerTargetState, float> > > > > const&) at /Users/alberli/Documents/git/opencv_contrib_track/opencv_contrib/modules/tracking/src/trackerStateEstimator.cpp:343
#9 0x000000010009831c in cv::TrackerStateEstimator::estimate(std::__1::vector<std::__1::vector<std::__1::pair<cv::Ptrcv::TrackerTargetState, float>, std::__1::allocator<std::__1::pair<cv::Ptrcv::TrackerTargetState, float> > >, std::__1::allocator<std::__1::vector<std::__1::pair<cv::Ptrcv::TrackerTargetState, float>, std::__1::allocator<std::1::pair<cv::Ptrcv::TrackerTargetState, float> > > > > const&) at /Users/alberli/Documents/git/opencv_contrib_track/opencv_contrib/modules/tracking/src/trackerStateEstimator.cpp:61
#10 0x0000000100088bac in cv::TrackerModel::runStateEstimator() at /Users/alberli/Documents/git/opencv_contrib_track/opencv_contrib/modules/tracking/src/trackerModel.cpp:117
#11 0x0000000100055608 in cv::TrackerBoostingImpl::updateImpl(cv::Mat const&, cv::Rect&) at /Users/alberli/Documents/git/opencv_contrib_track/opencv_contrib/modules/tracking/src/trackerBoosting.cpp:251
#12 0x0000000100051870 in cv::Tracker::update(cv::Mat const&, cv::Rect&) at /Users/alberli/Documents/git/opencv_contrib_track/opencv_contrib/modules/tracking/src/tracker.cpp:104

Crash at function:

const _Tp& Mat::at(int i0) const
{
...
    CV_DbgAssert(data);
...
}

data is NULL.

bool TrackerBoostingImpl::updateImpl( const Mat& image, Rect2d& boundingBox ):
image cv::Mat
flags int 1124024320
dims int 2
rows int 640
cols int 480

boundingBox Rect2d &
x double 130.33332824707031
y double 277.33332824707031
width double 106.33332824707031
height double 137.66667175292969

float StrongClassifierDirectSelection::eval( const Mat& response ):
response const cv::Mat & 0x0000000109575220
flags int 0
dims int 0
rows int 0
cols int 0
data uchar * NULL 0x0000000000000000

void Detector::classifySmooth( const std::vector<Mat>& images, float minMargin )
{
...
      m_confidences[curPatch] = m_classifier->eval( images[curPatch] );  // crash at this line

      // fill matrix
      m_confMatrix( row, col ) = m_confidences[curPatch];
...
}

images const std::__1::vector<cv::Mat, std::__1::allocatorcv::Mat > & size=9435 0x000000016e1b0b20

curPatch int 9435
row int 111
col int 0

[Alberl-Lee]

the KCF another crash error is:
OpenCV Error: Assertion failed (type == srcB.type() && srcA.size() == srcB.size()) in mulSpectrums, file /Users/alberli/Documents/git/opencv_contrib_track/modules/core/src/dxt.cpp, line 3546

stack is:
#1 0x00000001003e4b48 in cv::error(cv::Exception const&) at /Users/alberli/Documents/git/opencv_contrib_track/modules/core/src/system.cpp:661
#2 0x00000001003e3ebc in cv::error(int, cv::String const&, char const*, char const*, int) at /Users/alberli/Documents/git/opencv_contrib_track/modules/core/src/system.cpp:666
#3 0x000000010026f63c in cv::mulSpectrums(cv::_InputArray const&, cv::InputArray const&, cv::OutputArray const&, int, bool) at /Users/alberli/Documents/git/opencv_contrib_track/modules/core/src/dxt.cpp:3546
#4 0x00000001001c1e88 in cv::TrackerKCFImpl::updateImpl(cv::Mat const&, cv::Rect&) at /Users/alberli/Documents/git/opencv_contrib_track/opencv_contrib/modules/tracking/src/trackerKCF.cpp:415
#5 0x00000001001ac4c4 in cv::Tracker::update(cv::Mat const&, cv::Rect&) at /Users/alberli/Documents/git/opencv_contrib_track/opencv_contrib/modules/tracking/src/tracker.cpp:104

Crash at function:

void cv::mulSpectrums( InputArray _srcA, InputArray _srcB,
                       OutputArray _dst, int flags, bool conjB )
{
...
CV_Assert( type == srcB.type() && srcA.size() == srcB.size() );
...
}

type 14
srcB.type() 14
srcA.size() 134
srcB.size() 135

[Alberl-Lee]

I am use iphone6s plus.

[Alberl-Lee]

I fixed the fisrt crash at mat.inl.hpp, line 956:
change < to <=.

//    CV_DbgAssert((unsigned)i0 < (unsigned)size.p[0]);
//    CV_DbgAssert((unsigned)(i1 * DataType<_Tp>::channels) < (unsigned)(size.p[1] * channels()));

 CV_DbgAssert((unsigned)i0 <= (unsigned)size.p[0]); 
 CV_DbgAssert((unsigned)(i1 * DataType<_Tp>::channels) <= (unsigned)(size.p[1] * channels()));

The other two crash, I can't fixed.

[alalek]

Don't do this. Condition in assertion is valid and it detects memory corruption problems.

The bug is in this code:

    y=Mat::zeros((int)roi.height,(int)roi.width,CV_64F);
    for(unsigned i=0;i<roi.height;i++){
      for(unsigned j=0;j<roi.width;j++){
        y.at<double>(i,j)=(i-roi.height/2+1)*(i-roi.height/2+1)+(j-roi.width/2+1)*(j-roi.width/2+1);
      }
    }

Here roi is Rect2d (double type) with non integer values (like 106.33332824707031).

Both conditions should be changed:

• i<roi.height should be replaced i<(int)roi.height.
• similar change for j

Probably there are other similar places in this module.

[Alberl-Lee]

trackerKCF.cpp, i fixed like this:

1. Change
   Rect2d roi;
   to
   Rect roi;

2.Change

bool TrackerKCFImpl::initImpl( const Mat& image, const Rect2d& boundingBox ){
    frame=0;
    roi = boundingBox;

to

  bool TrackerKCFImpl::initImpl( const Mat& image, const Rect2d& boundingBox ){
    frame=0;
    roi = Rect((int)boundingBox.x, (int)boundingBox.y, (int)boundingBox.width, (int)boundingBox.height);

I'm use 3.2 version, so can't compile the last version, could you push the code ?