AndroidJUnit4.class + org.junit.Assume.assumeTrue = AssumptionViolatedException

Issue

I've managed to get my Android project transitioned over to JUnit4, and of course the main reason I wanted to do it isn't working. Would love any help if anyone's got ideas here.

The problem I'm trying to solve is that I want to automatically skip certain tests if the build is not pointed at the staging server. I've got this set up with a BUILD_TYPE which is using gradle to inject the base URL.

I set up an assumeThat clause in my setup which correctly identifies when the build is not staging, but instead of halting and ignoring the rest of the test, it throws an exception and fails.

Here's my base class for my live API tests - I've annotated descending from this with @RunWith(AndroidJUnit4.class), so in theory this should always be run with the JUnit4 runner:

package com.[my package].nonuitests.liveservertests;

import android.support.test.runner.AndroidJUnit4;
import com.[my package].nonuitests.BaseAndroidTest;
import org.junit.Test;
import org.junit.runner.RunWith;

/**
 * Tests against the live API. All tests descending from this class will
 * be ignored if the BUILD_TYPE is not staging.
 */
@RunWith(AndroidJUnit4.class)
public class BaseLiveServerTests extends BaseAndroidTest {

    private static final String STAGE = "staging";

    /******************
     * SETUP/TEARDOWN *
     ******************/

    @Override
    public void setUp() throws Exception {
        super.setUp();

        //TODO: Y U NO WORK?!
        //This should cause the rest of the test to be skipped if it fails,
        //but is instead throwing an AssumptionViolatedException. 
        assumeTrue(STAGE.equals(BuildConfig.BUILD_TYPE));
    }
}

So, my questions:

1. Is there a better way to do this? In theory this could be done with flavors, but I was trying that earlier and it made everything else way more complicated.
2. My research indicates there's some kind of thing that Google's not implementing in their runner that's causing this to bomb out, but I'm having a hell of a time figuring out what I can/should do to fix this. Any suggestions of things I should subclass to get this to work as expected?

Any other thoughts would be most appreciated. Thanks!

Edit (1/26/15 11:40am CST): Per Grzesuav's suggestion, I took a stab at implementing this as an @Rule, but at the moment it's still not working. This seems a promising path, but it ain't working at the moment.

Edit 2 (1/26/15 12:15pm CST): OK, now it's working.

Solution

https://github.com/junit-team/junit/wiki/Assumptions-with-assume

ad 2) Custom runners could differently treat assume statement. To fix it you should write own version of Android runner and implement a way of dealing with assumes as native JUnit runner does or make a bug for android test runner.

ad 1) Suggested by me : try use JUnit Rules :

http://www.codeaffine.com/2013/11/18/a-junit-rule-to-conditionally-ignore-tests/ http://cwd.dhemery.com/2010/12/junit-rules/

Answered By - Grzesuav