Kafka migration / handling "UNKNOWN_TOPIC_OR_PARTITION" necessary?

Issue

We use spring KafkaListener to listen topics. Everything works fine.

Now we have to migrate to an another Kafka Broker, that at the time we are deploying our application, does not serve any topics.

I simulated this locally and our listener receives endless UNKNOWN_TOPIC_OR_PARTITION warnings. What is somehow the expected behaviour.

Error while fetching metadata with correlation id 146 : {x-events-live=UNKNOWN_TOPIC_OR_PARTITION}

The question that I have is, if we have to handle those warnings in terms of memory leakage etc or if we could just wait until the topics are available.

@KafkaListener(
        id = "id",
        topics = ["topic"],
        groupId = "group",
        containerFactory = "myContainer",
        autoStartup = "false"
)
fun thingChnanged(@Payload thing: Thing,
                            record: ConsumerRecord<String, String>,
                            @Headers headers: MessageHeaders) {
    doSomething(thing)
}

Solution

There's isn't any more memory leaking than a regular consumer TCP connection.

Yes, you can wait, or you can add @Bean NewTopic to your app so the app itself will create the topic with the expected settings

Answered By - OneCricketeer