Cp-ansible: Configure Kafka broker to use Zookeeper chroot path

Created on 14 Jun 2021  ·  5Comments  ·  Source: confluentinc/cp-ansible

We need to configure Kafka brokers to use Zookeeper chroot path. By default, Kafka creates znodes in root of Zookeeper.
In order to separate app and Kafka configurations, it is necessary to set different chroot path for Kafka.

Is it possible to have Kafka with chroot Zookeeper path provisioned by cp-ansible?

As an example. we need cp-ansible to configure _zookeeper.connect_ to have a value like: hostname1:port1,hostname2:port2,hostname3:port3/chroot/path.

Most helpful comment

@a-narenji Thanks for the additional details. Unfortunately, at this time we do not support this configuration. Specifically around the scram users. Of additional note, is that Zookeeper is being removed as a kafka dependency soon, so we will review this and see if it makes sense to allow this modification with zookeeper for a future release.

All 5 comments

@a-narenji Are you using an existing zookeeper cluster for multiple applications? If so, we generally do not recommend this, we recommend you let cp-ansible install zookeeper and manage it. We cannot guarantee that whichever version you may already be running is compatible etc.

You can override all configurations via custom properties as per our documentation here:

https://docs.confluent.io/ansible/current/ansible-configure.html#set-custom-component-properties

So you could set the properties to your desired configuration on the Broker, however if we are not managing zookeeper for you, you would need to configure that yourself manually.

@JumaX Many thanks for your reply.
We are not using existing Zookeeper cluster, and we use the same Zookeeper which cp-ansible set it up for us.
Our develop team may need to use Zookeeper for another purpose, and it may not be reasonable to have a second Zookeeper cluster dedicated to develop team requirements.

So, it would be helpful that cp-ansible let us to have chroot path in _zookeeper.connect_ property.
Additionally, since we need to enable SASL, it is not enough to change _zookeeper.connect_ only, but also, it is required to change Create SCRAM Users and Create SCRAM 256 Users tasks in such a way that cp-ansible be able to create SCRAM users to use Zookeeper chroot path.

@a-narenji Thanks for the additional details. Unfortunately, at this time we do not support this configuration. Specifically around the scram users. Of additional note, is that Zookeeper is being removed as a kafka dependency soon, so we will review this and see if it makes sense to allow this modification with zookeeper for a future release.

@a-narenji Can you please review: https://github.com/confluentinc/cp-ansible/pull/694

@domenicbove We checked it in our test environment.
It worked great.
Many thanks for adding this feature.

Was this page helpful?
0 / 5 - 0 ratings

Related issues

luizm picture luizm  ·  18Comments

chuck-confluent picture chuck-confluent  ·  5Comments

Fobhep picture Fobhep  ·  12Comments

LGouellec picture LGouellec  ·  4Comments

Fobhep picture Fobhep  ·  7Comments