automatic topic creation fails with @PostConstruct

[TrentonAdams]

In what version(s) of Spring for Apache Kafka are you seeing this issue?

3.1.0 - though probably all as the main branch appears to be using SmartInitializingSingleton

Describe the bug

If I use @PostConstruct, and my bean depends on a NewTopic, and I've also created a KafkaAdmin bean, the topic is not automatically created.

I suspect the problem is the use of the SmartInitializingSingleton. It only runs after all beans have finished being initialized, and @PostConstruct methods have ran.

The workaround was to use a event listern on a context refreshed event, but I'm uncertain if that guarantees it's done before any business logic runs. I have to verify. But, the key point is that I've never encountered a problem like this with spring, it's very unexpected. At the very least, in the documentation for how topics are automatically created, it should state that it doesn't work with @PostConstruct.

To Reproduce

Use @PostConstruct, and depend on your NewTopic bean. You'll get a failure about the topic not existing.

Expected behavior

I was expecting, that because I depended on the NewTopic, it would be created, according to the documentation. This is true, that works, but not if my bean uses @PostConstruct

Sample

I don't have one, but I suspect it's easy enough for devs to replicate. If not, I can try and find the time to create one.

[artembilan]

Show, please, at least, some config, so we will know how to model the test to reproduce.
Thanks

[TrentonAdams]

I decided to just make a project...

https://github.com/TrentonAdams/spring-kafka-post-construct

docker compose up # in one terminal

In another terminal...
./mvnw clean install spring-boot:run -Dspring-boot.run.profiles=post-construct
./mvnw clean install spring-boot:run -Dspring-boot.run.profiles=application-ready

http://localhost:8080 for the kafka-ui

[artembilan]

    @PostConstruct
    public void init() {
        // This will fail because the topic hasn't been created yet
        // The @PostConstruct runs before SmartInitializingSingleton
        log.info("Attempting to send message to topic: {}", postConstructTopic.name());
        kafkaTemplate.send(postConstructTopic.name(), "test-message-from-post-construct");
    }

You cannot do heavy network communication work from the @PostConstruct.
It has to be really done when an application context is ready.
Or via the mentioned event, or implementing SmartLifecycle.

I've never encountered a problem like this with spring, it's very unexpected

Please, double check those usages, but any external services calls are not OK from @PostConstruct.
The application context is really not ready yet and that might not be a surprise that something has not been initialized yet, like in your case the KafkaAdmin has not initiated topic on the broker.
And that is exactly why we do that a bit later, because not all the beans in the application context are initialized when that @PostConstruct is invoked.

I'm not sure if we have anything in docs, but that is not Spring for Apache Kafka responsibility.
We don't owe that @PostConstruct, so no way that we may make any connection with what you are doing in your application and what this framework is responsible for.

[TrentonAdams]

Interesting. I read somewhere that @PostConstruct was the place to do that, as all dependant beans would be initialized, but normal operations would not be underway yet.

[artembilan]

Right, dependent beans are initialized, but in your case you'd like to have an entity in the external system which is a heavy network call.
And that is not recommended to do from @PostConstruct.

(Please, use back ticks around word starting with @, otherwise you invite that GH user to our discussion)

[TrentonAdams]

oh, oops, lol.

The spring docs don't seem to say anything about what is okay or not okay in there, so that's sad. :(

https://docs.spring.io/spring-framework/reference/core/beans/annotation-config/postconstruct-and-predestroy-annotations.html

Do you happen to know where this is mentioned?

[artembilan]

I cannot tell about other projects, but here is what I say for Spring Integration: https://docs.spring.io/spring-integration/reference/overview.html#programming-considerations

If your component is an InitializingBean or uses @PostConstruct methods, do not send any messages from these initialization methods.
The application context is not yet initialized when these methods are called, and sending such messages is likely to fail.
If you need to send a messages during startup, implement ApplicationListener and wait for the ContextRefreshedEvent.
Alternatively, implement SmartLifecycle, put your bean in a late phase, and send the messages from the start() method.

[TrentonAdams]

Yeah, I've confirmed that InitializingBean, '@PostConstruct', and SmartInitializingBean, do not work. The only thing that's worked is the context refreshed event, but I did not try SmartLifecycle, as that's a large interface, so it would tarnish my pristine class, lol.

So, that's fine, I'll have my app define an initialization interface with one method, and do context refreshed event.

Might be useful to include that in the docs as a one liner though.

Thanks for the responses! You can close now if you like.

[artembilan]

Thank you for confirmation!
Repurpose the to the documentation to add a NOTE into the https://docs.spring.io/spring-kafka/reference/4.0/kafka/sending-messages.html#kafka-template about sending messages when the application context is fully ready instead of @PostConstrcuct which is too early because of context initialization phase.