Why do tests fail when executed as a class?

Issue

I have a series of tests separated into classes. The tests run properly when run individually or when run as a whole package. Most classes also run properly when executed as a class, but there is one class that fails to execute with the following error:

java.lang.RuntimeException: Delegate runner 'org.robolectric.RobolectricTestRunner' for AndroidJUnit4 could not be loaded.

    at androidx.test.ext.junit.runners.AndroidJUnit4.throwInitializationError(AndroidJUnit4.java:92)
    at androidx.test.ext.junit.runners.AndroidJUnit4.loadRunner(AndroidJUnit4.java:82)
    at androidx.test.ext.junit.runners.AndroidJUnit4.loadRunner(AndroidJUnit4.java:51)
    at androidx.test.ext.junit.runners.AndroidJUnit4.<init>(AndroidJUnit4.java:46)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
    at org.junit.internal.builders.AnnotatedBuilder.buildRunner(AnnotatedBuilder.java:104)
    at org.junit.internal.builders.AnnotatedBuilder.runnerForClass(AnnotatedBuilder.java:86)
    at org.junit.runners.model.RunnerBuilder.safeRunnerForClass(RunnerBuilder.java:59)
    at org.junit.internal.builders.AllDefaultPossibilitiesBuilder.runnerForClass(AllDefaultPossibilitiesBuilder.java:26)
    at org.junit.runners.model.RunnerBuilder.safeRunnerForClass(RunnerBuilder.java:59)
    at org.junit.internal.requests.ClassRequest.getRunner(ClassRequest.java:33)
    at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:49)
    at com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:33)
    at com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:230)
    at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:58)
Caused by: java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
    at androidx.test.ext.junit.runners.AndroidJUnit4.loadRunner(AndroidJUnit4.java:72)
    ... 16 more
Caused by: java.lang.NoClassDefFoundError: androidx/test/rule/ActivityTestRule
    at java.lang.Class.getDeclaredMethods0(Native Method)
    at java.lang.Class.privateGetDeclaredMethods(Class.java:2701)
    at java.lang.Class.getDeclaredMethods(Class.java:1975)
    at org.junit.internal.MethodSorter.getDeclaredMethods(MethodSorter.java:54)
    at org.junit.runners.model.TestClass.scanAnnotatedMembers(TestClass.java:65)
    at org.junit.runners.model.TestClass.<init>(TestClass.java:57)
    at org.junit.runners.ParentRunner.createTestClass(ParentRunner.java:88)
    at org.junit.runners.ParentRunner.<init>(ParentRunner.java:83)
    at org.junit.runners.BlockJUnit4ClassRunner.<init>(BlockJUnit4ClassRunner.java:65)
    at org.robolectric.internal.SandboxTestRunner.<init>(SandboxTestRunner.java:56)
    at org.robolectric.RobolectricTestRunner.<init>(RobolectricTestRunner.java:92)
    ... 21 more
Caused by: java.lang.ClassNotFoundException: androidx.test.rule.ActivityTestRule
    at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:419)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:352)
    ... 32 more

The problem only appears in this specific class and when the tests are run on the class, not individually or as a package.

I have verified that only androidx libraries are being used, that rules are setup properly and that the roboelectric dependency is included as per other answers in this forum.

Has anyone had a similar problem or has any suggestions to fix this issue?

Note: I have tried recreating the issue in a small example code, but haven't been able to. I also can't post the code that fails due to proprietary reasons.

Solution

The problem was due to having public methods in the test class that were not annotated with @Test. The solution I used was adding @Ignore and @Test annotations to the method. ie:

@Ignore
@Test
fun someUnImplementedTest() {
}

Adding only @Test works as well, but it will run the test and mark it as passed allowing you to forget that it's not yet implemented.

Answered By - Santi