Kotlin from generateSequence() to flow() , but wrong bytecode generated

Issue

I have a function that uses http request to retrieve remote page. That page may has one or zero next page's link. If I want to generate a chain of all pages , generateSequence is an ideal solution. This is what I have done:

First , there are two utility functions :

fun getBlockingDocument(url: String): Document? , as the name suggests, it is a blocking function. The implementation is just sending HTTP request and parsing to a JSoup document.

fun getNextIndexPage(doc: Document, url: String): String? , it is also a blocking function , but it's not related to network , it just parses to get the next page, so blocking is OK here.

OK , here is the sequence code :

val initUrl = // initial url

generateSequence(tools.getBlockingDocument(initUrl).let { initUrl to it }) { (url, doc) ->
  doc?.let {
    parser.getNextIndexPage(doc, url)
  }?.let { nextUrl ->
    nextIndexUrl to tools.getBlockingDocument(nextUrl)
  }
}.forEachIndexed { index, urlAndDoc ->
  val url = urlAndDoc.first
  logger.info("[{}] : {}", index, url)
}

It works well , and correctly chains all pages.

But what if I change the network call to a suspend function ? This is what I created :

suspend fun getSuspendingDocument(url: String): Document?

I found no similar generateSequence builder samples as flow , So I implement like this :

  @ExperimentalCoroutinesApi
  @Test
  fun testGetAllPagesByFlow() {
    val flow = flow<Pair<String, Document?>> {
      suspend fun generate(url: String) {
        tools.getSuspendingDocument(url)?.let { url to it }?.also { (url, doc) ->
          emit(url to doc)

          parser.getNextIndexPage(doc, url)?.also { nextUrl ->
            generate(nextUrl) // recursive
          }
        }
      }
      generate("http://...initial url here")
    } // flow

    runBlocking {
      flow.collectIndexed { index, urlAndDoc ->
        val url = urlAndDoc.first
        logger.info("[{}] : {}", index, url)
      }
    }
  }

I uses a recursive call (fun generate()) to emit the next url found in each page. I am not sure if it is the idiomatic way to create a Flow but I found no similar codes. If you have better/idiomatic way , please tell me , thanks a lot!

Anyway , I think it should work , but my IDE (IntelliJ) complains wrong bytecode generated , which I've never seen it before.

Error:Kotlin: [Internal Error] org.jetbrains.kotlin.codegen.CompilationException: Back-end (JVM) Internal error: wrong bytecode generated
  @Lorg/jetbrains/annotations/Nullable;() // invisible
    // annotable parameter count: 1 (visible)
    // annotable parameter count: 1 (invisible)
    @Lorg/jetbrains/annotations/NotNull;() // invisible, parameter 0
   L0
   L1
    ALOAD 0
    GETFIELD destiny/data/FlowTest$testGetAllPagesByFlow$flow$1.p$ : Lkotlinx/coroutines/flow/FlowCollector;
    ASTORE 2
   L2
   L3
    LINENUMBER 44 L3
    NEW destiny/data/FlowTest$testGetAllPagesByFlow$flow$1$1
    DUP
    ALOAD 2
    ALOAD 3
    ACONST_NULL
    INVOKESPECIAL destiny/data/FlowTest$testGetAllPagesByFlow$flow$1$1.<init> (Lkotlinx/coroutines/flow/FlowCollector;Ldestiny/data/FlowTest$testGetAllPagesByFlow$flow$1$1;Lkotlin/coroutines/Continuation;)V
    ASTORE 3
   L4
   L5
    LINENUMBER 56 L5
    ALOAD 3
    CHECKCAST destiny/data/FlowTest$testGetAllPagesByFlow$flow$1$1
    ALOAD 0
    GETFIELD destiny/data/FlowTest$testGetAllPagesByFlow$flow$1.$initUrl : Ljava/lang/String;
    ALOAD 0
    ALOAD 0
    ALOAD 2
    PUTFIELD destiny/data/FlowTest$testGetAllPagesByFlow$flow$1.L$0 : Ljava/lang/Object;
    ALOAD 0
    ALOAD 3
    PUTFIELD destiny/data/FlowTest$testGetAllPagesByFlow$flow$1.L$1 : Ljava/lang/Object;
    ALOAD 0
    ICONST_1
    PUTFIELD destiny/data/FlowTest$testGetAllPagesByFlow$flow$1.label : I
    INVOKEVIRTUAL destiny/data/FlowTest$testGetAllPagesByFlow$flow$1$1.invoke (Ljava/lang/String;Lkotlin/coroutines/Continuation;)Ljava/lang/Object;
   L6
    DUP
    ALOAD 4
    IF_ACMPNE L7
   L8
    LINENUMBER 42 L8
    ALOAD 4
    ARETURN
   L9
    ALOAD 0
    GETFIELD destiny/data/FlowTest$testGetAllPagesByFlow$flow$1.L$1 : Ljava/lang/Object;
    CHECKCAST destiny/data/FlowTest$testGetAllPagesByFlow$flow$1$1
    ASTORE 3
    ALOAD 0
    GETFIELD destiny/data/FlowTest$testGetAllPagesByFlow$flow$1.L$0 : Ljava/lang/Object;
    CHECKCAST kotlinx/coroutines/flow/FlowCollector
    ASTORE 2
   L10
    ALOAD 1
    INVOKESTATIC kotlin/ResultKt.throwOnFailure (Ljava/lang/Object;)V
    ALOAD 1
   L7
    LINENUMBER 58 L7
    POP
   L11
   L12
    GETSTATIC kotlin/Unit.INSTANCE : Lkotlin/Unit;
    ARETURN
   L13
   L14
   L15
    NEW java/lang/IllegalStateException
    DUP
    LDC "call to 'resume' before 'invoke' with coroutine"
    INVOKESPECIAL java/lang/IllegalStateException.<init> (Ljava/lang/String;)V
    ATHROW
    RETURN
   L16
    LOCALVARIABLE $this$flow Lkotlinx/coroutines/flow/FlowCollector; L2 L14 2
    LOCALVARIABLE $fun$generate$1 Ldestiny/data/FlowTest$testGetAllPagesByFlow$flow$1$1; L4 L11 3
    LOCALVARIABLE this Ldestiny/data/FlowTest$testGetAllPagesByFlow$flow$1; L0 L13 0
    LOCALVARIABLE $result Ljava/lang/Object; L0 L13 1
    MAXSTACK = 5
    MAXLOCALS = 4
File being compiled at position: (42,46) in /destiny/data/core/src/test/java/destiny/data/FlowTest.kt
The root cause org.jetbrains.kotlin.codegen.CompilationException was thrown at: org.jetbrains.kotlin.codegen.TransformationMethodVisitor.visitEnd(TransformationMethodVisitor.kt:92)
    at org.jetbrains.kotlin.codegen.FunctionCodegen.endVisit(FunctionCodegen.java:990)
    at org.jetbrains.kotlin.codegen.FunctionCodegen.generateMethodBody(FunctionCodegen.java:487)
    at org.jetbrains.kotlin.codegen.FunctionCodegen.generateMethod(FunctionCodegen.java:260)
    at org.jetbrains.kotlin.codegen.FunctionCodegen.generateMethod(FunctionCodegen.java:176)
    ...

It's very long , the remainings are omitted.

What's wrong with this code ?

And if the recursive way is not ideal , is there any better solution ? (like generateSequence , it is beautiful) . Thanks.

Environments :

<kotlin.version>1.3.50</kotlin.version>
<kotlin.compiler.jvmTarget>1.8</kotlin.compiler.jvmTarget>
<dependency>
  <groupId>org.jetbrains.kotlinx</groupId>
  <artifactId>kotlinx-coroutines-core</artifactId>
  <version>1.3.2</version>
</dependency>
IntelliJ 2018.3.6

$ java -version
java version "11.0.3" 2019-04-16 LTS
Java(TM) SE Runtime Environment 18.9 (build 11.0.3+12-LTS)
Java HotSpot(TM) 64-Bit Server VM 18.9 (build 11.0.3+12-LTS, mixed mode)

Solution

Why use recursion at all? The following code would do the same

val f: Flow<Pair<String, Document?>> = flow {
    var nextUrl: String? = url

    while (nextUrl != null) {
        val doc = tools.getSuspendingDocument(nextUrl)
        emit(url to doc)

        if (doc == null) break;

        nextUrl = parser.getNextIndexPage(doc, url)

    }
}

Answered By - Evgeny Bovykin