Why am I getting a local reference table overflow?

Issue

I'm trying to figure out how to resolve a crash happening on my code when the number of cue points is high. My test reproduce it every time with a number of 530 cue items. Each cue has a dataSize of 1.

You can see my code below. Essentially native_update_cue_points is called from the java layer. This triggers multiple calls to the java layer from jni layer. Somehow this is causing a table overflow. I'm thinking somehow each call to the static java function is not releasing the allocated string values causing the stack overflow?

Sample code

static void native_update_cue_points(JNIEnv *env, jclass clazz)
{
    if (smpMediaPlayer)
    {
        std::list<Cue> cuePoints;

        smpMediaPlayer->getCuePoints(&cuePoints);

        for (std::list<Cue>::iterator it = cuePoints.begin(); it != cuePoints.end(); it++)
        {
            Cue cue = *it;

            java_update_cue_point(env, clazz, smFields.global_ref_thiz, 0, &cue);
        }
    }
}

static void java_update_cue_point (JNIEnv *env, jclass clazz, jobject thiz, int reqType, Cue *pCue)
{
    if (!pCue) {
        return;
    }

    jlong id = pCue->id;
    jint type = pCue->type;
    jint extra = pCue->extra;
    jlong pos = pCue->pos;
    jlong duration = pCue->duration;
    jobjectArray jkeys = NULL;
    jobjectArray jvalues = NULL;

    int dataSize = pCue->data.size();

    if (0 < dataSize)
    {
        jkeys = env->NewObjectArray(dataSize, env->FindClass("java/lang/String"), NULL);
        jvalues = env->NewObjectArray(dataSize, env->FindClass("[B"), NULL);

        int position = 0;
        for (std::map<std::string, std::string>::iterator it = pCue->data.begin(); it != pCue->data.end(); it++)
        {
            // set key
            env->SetObjectArrayElement(jkeys, position, env->NewStringUTF(it->first.c_str()));

            // set value
            jbyteArray byteArray = env->NewByteArray(it->second.length());
            env->SetByteArrayRegion(byteArray, 0, it->second.length(), (const signed char *)it->second.c_str());
            env->SetObjectArrayElement(jvalues, position, byteArray);
            env->DeleteLocalRef(byteArray);

            position++;
        }
    }

    // report update
    env->CallStaticVoidMethod(
                clazz,
                smFields.native_callback_add_cue_point,
                thiz,
                (jint)reqType, id, type, extra, pos, duration, jkeys, jvalues);
}

Stacktrace

JNI ERROR (app bug): local reference table overflow (max=512)
 local reference table dump:
   Last 10 entries (of 512):
       511: 0x13054090 java.lang.String[] (1 elements)
       510: 0x70732980 java.lang.Class<java.lang.String>
       509: 0x130517c0 java.lang.String "song_artist"
       508: 0x13054080 byte[][] (1 elements)
       507: 0x7079af18 java.lang.Class<byte[]>
       506: 0x13054070 java.lang.String[] (1 elements)
       505: 0x70732980 java.lang.Class<java.lang.String>
       504: 0x13051700 java.lang.String "song_artist"
       503: 0x12d17ff0 byte[][] (1 elements)
       502: 0x7079af18 java.lang.Class<byte[]>
   Summary:
       205 of java.lang.Class (2 unique instances)
       103 of java.lang.String[] (1 elements) (103 unique instances)
       102 of java.lang.String (102 unique instances)
       102 of byte[][] (1 elements) (102 unique instances)  

---

UPDATE 1:
I did a bit more digging and was able to narrow it down to the call FindClass. This leaves me even more confused! :/ Here is my simplified native_update_cue_point method. Do I need to release somehow the FindClass call!?

Code

static void java_update_cue_point (JNIEnv *env, jclass clazz, jobject thiz, int reqType, Cue *pCue)
{
    jclass jstringClass = env->FindClass("java/lang/String");
}

Stacktrace

JNI ERROR (app bug): local reference table overflow (max=512)
 local reference table dump:
   Last 10 entries (of 512):
       511: 0x70732980 java.lang.Class<java.lang.String>
       510: 0x70732980 java.lang.Class<java.lang.String>
       509: 0x70732980 java.lang.Class<java.lang.String>
       508: 0x70732980 java.lang.Class<java.lang.String>
       507: 0x70732980 java.lang.Class<java.lang.String>
       506: 0x70732980 java.lang.Class<java.lang.String>
       505: 0x70732980 java.lang.Class<java.lang.String>
       504: 0x70732980 java.lang.Class<java.lang.String>
       503: 0x70732980 java.lang.Class<java.lang.String>
       502: 0x70732980 java.lang.Class<java.lang.String>
   Summary:
       512 of java.lang.Class (1 unique instances)  

---

UPDATE 2:
I was able to fix the table overflow by doing a DeleteLocalRef on every local object regardless if the java VM would take care of it. My thoughts are that the VM is accumulating items on the stack which will release after executing. I need to remove each item explicitly from the stack to avoid unwanted accumulations for later removal while on a large loop.

Still I'm left feeling like I haven't understood completely why this is happening and why my solution solves it.

Fixed code

static void java_update_cue_point (JNIEnv *env, jclass clazz, jobject thiz, int reqType, Cue *pCue)
{
    if (!pCue) {
        return;
    }

    jclass jstringClass = env->FindClass("java/lang/String");
    jclass jbyteArrayClass = env->FindClass("[B");
    jlong id = pCue->id;
    jint type = pCue->type;
    jint extra = pCue->extra;
    jlong pos = pCue->pos;
    jlong duration = pCue->duration;
    jobjectArray jkeys = NULL;
    jobjectArray jvalues = NULL;

    int dataSize = pCue->data.size();

    if (0 < dataSize)
    {
        jkeys = env->NewObjectArray(dataSize, jstringClass, NULL);
        jvalues = env->NewObjectArray(dataSize, jbyteArrayClass, NULL);

        int position = 0;
        for (std::map<std::string, std::string>::iterator it = pCue->data.begin(); it != pCue->data.end(); it++)
        {
            // set key
            jstring jkey = env->NewStringUTF(it->first.c_str());
            env->SetObjectArrayElement(jkeys, position, jkey);
            env->DeleteLocalRef(jkey); // FIX

            // set value
            jbyteArray byteArray = env->NewByteArray(it->second.length());
            env->SetByteArrayRegion(byteArray, 0, it->second.length(), (const signed char *)it->second.c_str());
            env->SetObjectArrayElement(jvalues, position, byteArray);
            env->DeleteLocalRef(byteArray);

            position++;
        }
    }

    // report update
    env->CallStaticVoidMethod(
                clazz,
                smFields.native_callback_add_cue_point,
                thiz,
                (jint)reqType, id, type, extra, pos, duration, jkeys, jvalues);

    if (jkeys) {
        env->DeleteLocalRef(jkeys); // FIX
    }

    if (jvalues) {
        env->DeleteLocalRef(jvalues); // FIX
    }

    if (jstringClass) {
        env->DeleteLocalRef(jstringClass); // FIX
    }

    if (jbyteArrayClass) {
        env->DeleteLocalRef(jbyteArrayClass); // FIX
    }
}

Solution

You're creating too many local references (two for classes, two for object arrays and some more for strings and byte arrays per run of java_update_cue_point). The VM can only handle a limited amount of local references. See "Referencing Java Objects" for some documentation. The "Global and Local References" section in the JNI function documentation also has some details. The VM will free local references automatically when a native method returns, but in a loop like yours you will have to delete references you don't need anymore with DeleteLocalRef.

To optimize a little, my recommendation would be to do the two FindClass calls outside of the loop in native_update_cue_points and pass them as parameters to java_update_cue_point. This way you'll only have two references for the class objects that the VM will free for you when native_update_cue_points returns, and you save processing time by not finding the same classes over and over again.

Answered By - user2543253