Android: flaky ViewModel unit tests with coroutines

Issue

I have a VM such as

class CityListViewModel(private val repository: Repository) : ViewModel() {
    @VisibleForTesting
    val allCities: LiveData<Resource<List<City>>> =
        liveData(context = viewModelScope.coroutineContext + Dispatchers.IO) {
            emit(Resource.Loading())
            emit(repository.getCities())
        }
}

And my tests are:

@ExperimentalCoroutinesApi
class CityListViewModelTest {
    @get:Rule
    val rule = InstantTaskExecutorRule()
    @get:Rule
    val coroutineTestRule = CoroutinesTestRule()

    @Test
    fun `allCities should emit first loading and then a Resource#Success value`() =
        runBlockingTest {
            val fakeSuccessResource = Resource.Success(
                listOf(
                    City(
                        1,
                        "UK",
                        "London",
                        Coordinates(34.5, 56.2)
                    )
                )
            )
            val observer: Observer<Resource<List<City>>> = mock()
            val repositoryMock: Repository = mock()

            val sut =
                CityListViewModel(repositoryMock)
            doAnswer { fakeSuccessResource }.whenever(repositoryMock).getCities()

            sut.allCities.observeForever(observer)
            sut.allCities
            val captor = argumentCaptor<Resource<List<City>>>()
            captor.run {
                verify(observer, times(2)).onChanged(capture())
                assertEquals(fakeSuccessResource.data, lastValue.data)
            }
        }

    @Test
    fun `allCities should emit first loading and then a Resource#Error value`() =
        runBlockingTest {
            val fakeErrorResource = Resource.Error<List<City>>("Error")
            val observer: Observer<Resource<List<City>>> = mock()
            val repositoryMock: Repository = mock()

            val sut =
                CityListViewModel(repositoryMock)
            doAnswer { fakeErrorResource }.whenever(repositoryMock).getCities()

            sut.allCities.observeForever(observer)
            sut.allCities
            val captor = argumentCaptor<Resource<List<City>>>()
            captor.run {
                verify(observer, times(2)).onChanged(capture())
                assertEquals(fakeErrorResource.data, lastValue.data)
            }
        }
}

The problem I have is that the tests are very flaky: sometimes they both pass, sometimes one fails, but I can't seem to find out the problem.

Thanks!

Solution

The issue is that in the test, you don't have control over the IO Dispatcher. I'm assuming your CoroutinesTestRule is something like this Gist? This only overrides Dispatchers.Main, but your CityListViewModel uses Dispatchers.IO.

There are a few different options:

1. In CityListViewModel, you can avoid using Dispatchers.IO explicitly, and instead just rely on the viewModelScope which defaults to Dispatchers.Main. In your real Repository implementation, ensure that your suspending getCities() method redirects to Dispatchers.IO, i.e.

suspend fun getCities(): List<City> {
  withContext(Dispatchers.IO) {
    // do work
    return cities
  }
}

And in the CityListViewModel:

  val allCities: LiveData<Resource<List<City>>> = 
    liveData(context = viewModelScope.coroutineContext) {
      emit(Resource.Loading())
      emit(repository.getCities())
    }

In this case, things will continue to work as they do currently, and in your test, the mock Repository will just immediately return a value.

2. Inject Dispatchers.IO instead. If you're using a DI framework such as Dagger this will be easier, but you could essentially do something like:

class CityListViewModel(
  private val ioDispatcher: CoroutineDispatcher = Dispatchers.IO,
  private val repository: Repository
) : ViewModel() {
  @VisibleForTesting
  val allCities: LiveData<Resource<List<City>>> = 
    liveData(context = viewModelScope.coroutineContext + ioDispatcher) {
      emit(Resource.Loading())
      emit(repository.getCities())
    }
}

And then in your test:

val viewModel = CityListViewModel(
  ioDispatcher = TestCoroutineDispatcher(),
  repository = repository
)

Either of these should make your tests deterministic. If you are using Dagger, then I'd recommend doing both (create a production module to provide the Main, IO, and Default dispatchers, but have a test module that provides instances of TestCoroutineDispatcher instead), but also doing option 1 which is to make sure your suspending functions direct the work to another dispatcher if they're doing blocking work.

Answered By - Kevin Coppock