In Confluent Slack
Normally, to changes your number of open files you have to modify your kernel params (fs.file-nr) and also your /etc/security/limits.conf. With systemd, there is another step. You must modify the *.service file as well for the daemons to pick up the new ulimit params. This post shows the directives that apply. Once I added LimitNOFILE to the delivered
/lib/systemd/system/confluent-\*.servicefile then/proc/<pid>/limitsshowed the java daemon recognizing the open file limit. There probably should be either a documentation note or a change to the delivered systemd file since that delivered file will now need to be managed on every upgrade
https://confluentcommunity.slack.com/archives/C49R61XMM/p1537208485000100
Refer - https://unix.stackexchange.com/questions/345595/how-to-set-ulimits-on-service-with-systemd
OneCricketeer
All 7 comments
Ideally we'd have good enough defaults for LimitNOFile and such to not have to be concerned with this but adding an ability to override properties in the systemd service makes sense. Should be easy enough to add to the templates for all services e.g. for the broker
cotedm
on 17 Sep 2018
It can be done in playbook at _https://github.com/confluentinc/cp-ansible/blob/5.1.x/roles/confluent.kafka-broker/defaults/main.yml#L14_ , also increase or decrease the memory, for example
environment:
KAFKA_HEAP_OPTS: "-Xmx24g"
LimitNOFILE: 102400
jvalderrama
on 20 Jun 2019
That won't template the systemd file appropriately, though.
[Service]
Environment=LimitNOFILE=102400
Is different from
[Service]
LimitNOFILE=102400
You are right the correct implementation is here, pending to be reviewed:
https://github.com/confluentinc/cp-ansible/pull/107
jvalderrama
on 20 Jun 2019
This is great, been trying to find the right way to address this config issue for way too long
steevo590
on 30 Jul 2019
Quick update that we have a new release shipping shortly which will address this.
JumaX
on 8 Aug 2019
We have just merged the following PR which addresses the no of open files:
JumaX
on 23 Aug 2019
Related issues
LGouellec
·
4Comments
Fobhep
·
7Comments
luizm
·
18Comments
Fobhep
·
12Comments
a-narenji
·
5Comments