NoClassDefFoundError when running Espresso tests on Firebase

Issue

I've got a suite of instrumentation tests that are meant to be run using the Firebase Test Lab on various physical devices from Android 4.4 through 8.

I can successfully run these tests locally on any OS version as well as on newer devices in the Firebase Test Lab, however when I run these tests in the Firebase Test Lab on devices running 4.4 (Currently running on a Moto X and a Galaxy S4 Mini), an exception is reported, even though the individual test cases all report success. Here's the exception I'm seeing:

java.lang.NoClassDefFoundError: org.junit.internal.TextListener FATAL EXCEPTION: Instr: android.support.test.runner.AndroidJUnitRunner Process: today.onedrop.android.debug, PID: 5579 java.lang.NoClassDefFoundError: org.junit.internal.TextListener
    at android.support.test.internal.runner.listener.InstrumentationResultPrinter.instrumentationRunFinished(InstrumentationResultPrinter.java:221)
    at android.support.test.internal.runner.TestExecutor.reportRunEnded(TestExecutor.java:92)
    at android.support.test.internal.runner.TestExecutor.execute(TestExecutor.java:67)  
    at android.support.test.runner.AndroidJUnitRunner.onStart(AndroidJUnitRunner.java:375)  
    at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:1792)

I suspect that Firebase is actually not a factor and that these tests would fail in the same way if I had a Moto X or a Galaxy S4 Mini to test on locally. Unfortunately I can't test that theory.

After updating my Espresso tests to use TestOrchestrator, I began getting a NoClassDefFoundError on Android 4.4 local AVD's as well. It's not the same one as above though:

FATAL EXCEPTION: Instr: android.support.test.runner.AndroidJUnitRunner
Process: today.onedrop.android.debug, PID: 15683
java.lang.NoClassDefFoundError: org.junit.runner.Request
    at android.support.test.runner.AndroidJUnitRunner.onStart(AndroidJUnitRunner.java:353)
    at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:1701)

Using TestOrchestrator on Firebase produces the new error instead of the old one as well. It's anyone's guess as to whether they share the same root cause, but their similarities definitely seem suspicious.

UPDATE #2

@MartinZeitler's instincts about MultiDex appear to have been right. I've resolved the second NoClassDefFoundError both issues by adding the missing classes to a multidex keep file in my gradle config:

multiDexKeepFile file('multidex-config.txt')

multidex-config.txt:

org/junit/runner/Request.class
org/junit/internal/TextListener.class

It would seem that multidex on 4.4 needs some help. Or maybe this is just a hack that will just lead to random crashes with other missing classes. In any case everything is working again!

Solution

did you add a dependency to jUnit? even on elder versions of Android, that library class should be known... leading to the assumption that you might also need to set multiDexEnabled true in the build.gradle, when building for Android < API 22 (which is enabled by default on all later API).

dependencies {
    testImplementation 'junit:junit:4.12'
}

gcloud firebase test android models list lists devices...

┌───────────────────┬──────────┬─────────────────────────────────────┬──────────┬─────────────┬────────────────┬────────────┐
│      MODEL_ID     │   MAKE   │              MODEL_NAME             │   FORM   │  RESOLUTION │ OS_VERSION_IDS │    TAGS    │
├───────────────────┼──────────┼─────────────────────────────────────┼──────────┼─────────────┼────────────────┼────────────┤
│ serranolte        │ Samsung  │ Galaxy S4 mini                      │ PHYSICAL │  960 x 540  │ 19             │            │
│ victara           │ Motorola │ Moto X                              │ PHYSICAL │ 1920 x 1080 │ 19             │            │
└───────────────────┴──────────┴─────────────────────────────────────┴──────────┴─────────────┴────────────────┴────────────┘

gcloud firebase test android models describe serranolte

brand: Samsung
codename: serranolte
form: PHYSICAL
supportedVersionIds:
- '19'

gcloud firebase test android models describe victara

brand: Motorola
codename: victara
form: PHYSICAL
supportedVersionIds:
- '19'

it's both hardware devices.

Answered By - Martin Zeitler