Staging instance, all changes can be removed at any time

Skip to content

Add extra configuration to kafka consumer from journal.client

Antoine Lambert requested to merge generated-differential-D1358-source into generated-differential-D1358-target

While playing with kafka components in the docker-compose environment, I often stumbled across the issue described in T1575. Setting the max_poll_records configuration entry to the maximum number of messages to process in a batch fixes that issue.

I also observed that applications can hang while iterating on a consumer for messages. Indeed, default kafka behavior is to wait undefinitely for new messages while iterating. This can result in some messages being consumed but not processed (typically when the maximum number of messages to process is not reached) until new messages to consume are available. Setting a value for the consumer_timeout_ms configuration entry ensures that application will not hang when such a situation happens.

Related T1575

Migrated from D1358 (view on Phabricator)

Merge request reports