Need Explanation for OpenCV Android Behavior

Issue

I am trying to write an image stitching app for Android. I am using Android NDK for the native part of OpenCV. There are 3 different behaviors that should not happen and I would love any explanation as to why they do happen.

1. Only some images (from the same camera on device / same resolution) do not crash. The error when it does crash is below my C++ code.

2. The result of the image stitch looks like it just barrels one image. (I get this result about 20% of the time while it crashes the other 80%). I think this has something to do with resize line in the for loop. The example from the book divided the columns and rows by 10. When I do that, the image is only slightly barreled but very very pixelated. Again, it looks like only one image in this case too.

3. If I don't set the stitcher settings like this:

   stitcher.setRegistrationResol(-1); /// 0.6
   stitcher.setSeamEstimationResol(-1);   /// 0.1
   stitcher.setCompositingResol(-1);   //1
   stitcher.setPanoConfidenceThresh(-1);   //1
   stitcher.setWaveCorrection(true);
   stitcher.setWaveCorrectKind(detail::WAVE_CORRECT_HORIZ);
   

   the output image will be empty. This is weird to me because the example in the book works fine without them.

I have been using Chapter 6 from this book for the C++ portion of my project. Here is my C++ code:

#include <jni.h>
#include "aaron_picstitch_MyNDK.h"
#include "opencv2/highgui/highgui.hpp"
#include "opencv2/imgproc/imgproc.hpp"
#include <opencv2/stitching/stitcher.hpp>
#include <opencv2/core/mat.hpp>

#include <vector>
#include <android/log.h>

using namespace std;
using namespace cv;

char FILEPATH[100] = "/storage/emulated/0/PicStitch/cppResult.jpg";
//char FILEPATH1[100] = "/storage/emulated/0/PicStitch/cppTesta.jpg";
//char FILEPATH2[100] = "/storage/emulated/0/PicStitch/cppTestb.jpg";

JNIEXPORT void JNICALL Java_aaron_picstitch_CameraActivity_stitchImages(JNIEnv *env, jobject , jobjectArray images, jint size, jlong panoAddr)
{

       vector <Mat> imgs = vector<Mat>();
       Mat pano = Mat();
       Mat temp = Mat();
       Mat &srcRes = *(Mat *)panoAddr, img;

       jclass clazz = (env)->FindClass("org/opencv/core/Mat");
       jmethodID getNativeObjAddr = (env)->GetMethodID(clazz, "getNativeObjAddr", "()J");

       __android_log_print(ANDROID_LOG_VERBOSE, "IN CPP TESTTEST", "ADDR: %lld", panoAddr);

       int i = 0;
       for (i = 0; i < size; i++)
       {
            jobject obj = (env->GetObjectArrayElement(images, i));
            jlong result = (env)->CallLongMethod(obj, getNativeObjAddr, NULL);
            img = *(Mat *)result;
            resize(img, temp, Size(img.rows/2, img.cols/2));
            imgs.push_back(temp);
            env->DeleteLocalRef(obj);
       }
       env->DeleteLocalRef(images);

       Stitcher stitcher = Stitcher::createDefault();
       __android_log_print(ANDROID_LOG_VERBOSE, "IN CPP", "HERE 1 temp rows is: %d", temp.rows);

       stitcher.setRegistrationResol(-1); /// 0.6
       stitcher.setSeamEstimationResol(-1);   /// 0.1
       stitcher.setCompositingResol(-1);   //1
       stitcher.setPanoConfidenceThresh(-1);   //1
       stitcher.setWaveCorrection(true);
       stitcher.setWaveCorrectKind(detail::WAVE_CORRECT_HORIZ);

        __android_log_print(ANDROID_LOG_VERBOSE, "IN CPP", "Right before .stitch");
       Stitcher::Status status = stitcher.stitch(imgs, pano);
       __android_log_print(ANDROID_LOG_VERBOSE, "IN CPP", "HERE 2 Pano rows is : %d", pano.rows);

       if (status == Stitcher::OK)
       {
         __android_log_print(ANDROID_LOG_VERBOSE, "IN CPP", "STITCHING SHOULD WORK");
       }


       //pano.copyTo(srcRes);

       imwrite(FILEPATH, pano);

}

Here is the error from bullet #1:

04-22 20:51:47.192 32115-32651/aaron.picstitch E/cv::error(): OpenCVError Assertion failed (s >= 0) in void cv::setSize(cv::Mat&, int, int const*, const size_t*, bool), file /hdd2/buildbot/slaves/slave_ardbeg1/50-SDK/opencv/modules/core/src/matrix.cpp, line 116
04-22 20:51:47.192 32115-32651/aaron.picstitch A/libc: Fatal signal 6 (SIGABRT), code -6 in tid 32651 (AsyncTask #1)

Another weird problem that is not as big of an issue is that I use getNativeObjAddr() for the Mat object in the Java portion of the project, so I can put the result into it, but I get a segfault whenever I try to access it. Not sure why that happens but that issue can be worked around.

Any thoughts as to my problems are appreciated!

Solution

I did not figure out the solution to my problem. Instead, I rewrote a lot of the code using code from this example and then everything magically worked. I still don't know what was wrong but at least I have working code.

Answered By - ayrunn