Kafka-backup: ๋‹ค์–‘ํ•œ ๊ตฌ์„ฑ์— ๋Œ€ํ•œ ์˜ˆ ์ œ๊ณต

์— ๋งŒ๋“  2020๋…„ 01์›” 17์ผ  ยท  5์ฝ”๋ฉ˜ํŠธ  ยท  ์ถœ์ฒ˜: itadventurer/kafka-backup

  • [ ] ์ผ๋ฐ˜ ํ…์ŠคํŠธ
  • [ ] TLS
documentation help wanted

๊ฐ€์žฅ ์œ ์šฉํ•œ ๋Œ“๊ธ€

๋‹ค์Œ์€ SASL_SCRAM์„ ์‚ฌ์šฉํ•˜์—ฌ ์•”ํ˜ธํ™”๋œ ํด๋Ÿฌ์Šคํ„ฐ์— ์—ฐ๊ฒฐํ•˜๋Š” ๋ฐฉ๋ฒ•์— ๋Œ€ํ•œ ์˜ˆ์ž…๋‹ˆ๋‹ค.

name = ConnectorName
connector.class = de.azapps.kafkabackup.sink.BackupSinkConnector
tasks.max = 1
key.converter = org.apache.kafka.connect.converters.ByteArrayConverter
value.converter = org.apache.kafka.connect.converters.ByteArrayConverter
header.converter = org.apache.kafka.connect.converters.ByteArrayConverter
target.dir = /backups
max.segment.size.bytes = 1073741824
cluster.bootstrap.servers = hostname-1.domain:9092,hostname-2.domain:9092,hostname-3.domain:9092
cluster.sasl.jaas.config = org.apache.kafka.common.security.scram.ScramLoginModule required username="REDACTED" password="REDACTED";
cluster.sasl_mechanism = SCRAM-SHA-256
cluster.security.protocol = SASL_SSL
cluster.sasl.mechanism = SCRAM-SHA-256
cluster.ssl.keystore.location = /path/to/ss/kafka.server.keystore.jks
cluster.ssl.keystore.password = REDACTED
cluster.ssl.key.password = REDACTED
cluster.ssl.truststore.location = /path/to/ss/kafka.server.truststore.jks
cluster.ssl.truststore.password = REDACTED

ํ•ด๋‹น ๋ผ์ธ์—์„œ jaas ๊ตฌ์„ฑ์„ ์ œ๊ฑฐํ•˜๊ณ  ๋‹ค๋ฅธ ๊ณณ์— ๋ฐฐ์น˜ํ•  ๊ฒƒ์ด์ง€๋งŒ ๋‹ค๋ฅธ ์‚ฌ๋žŒ๋“ค์„ ์˜ฌ๋ฐ”๋ฅธ ๊ธธ๋กœ ์ด๋„๋Š” ๋ฐ ๋„์›€์ด ๋˜๋„๋ก ๊ตฌ์„ฑ์„ ๊ณต์œ ํ•˜๋Š” ๊ฒƒ์ด ์˜ณ๋‹ค๊ณ  ์ƒ๊ฐํ–ˆ์Šต๋‹ˆ๋‹ค.

๋ชจ๋“  5 ๋Œ“๊ธ€

์—ฐ๊ฒฐ ๋…๋ฆฝ ์‹คํ–‰ํ˜• ๊ตฌ์„ฑ์˜ ์˜ˆ?

docker๋ฅผ ์‚ฌ์šฉํ•˜์—ฌ sasl_scram์œผ๋กœ ๋ณด์•ˆ ํด๋Ÿฌ์Šคํ„ฐ์— ์—ฐ๊ฒฐํ•˜๊ธฐ ์œ„ํ•œ ์ƒ˜ํ”Œ ์„ค์ •๋„ ์žˆ์Šต๋‹ˆ๋‹ค. #10 ์— ์–ธ๊ธ‰๋˜์–ด ์žˆ๋‹ค๋Š” ๊ฒƒ์„ ์•Œ๊ณ  ์žˆ์ง€๋งŒ ๋‚ด ๊ตฌ์„ฑ์„ ์ œ๊ณตํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.

์˜ˆ, ์—ฌ๊ธฐ ๋ฌธ์ œ์— ๋ถ™์—ฌ๋„ฃ๊ธฐ๋งŒ ํ•˜๋ฉด ๋ฉ๋‹ˆ๋‹ค. :)

๋‹ค์Œ์€ SASL_SCRAM์„ ์‚ฌ์šฉํ•˜์—ฌ ์•”ํ˜ธํ™”๋œ ํด๋Ÿฌ์Šคํ„ฐ์— ์—ฐ๊ฒฐํ•˜๋Š” ๋ฐฉ๋ฒ•์— ๋Œ€ํ•œ ์˜ˆ์ž…๋‹ˆ๋‹ค.

name = ConnectorName
connector.class = de.azapps.kafkabackup.sink.BackupSinkConnector
tasks.max = 1
key.converter = org.apache.kafka.connect.converters.ByteArrayConverter
value.converter = org.apache.kafka.connect.converters.ByteArrayConverter
header.converter = org.apache.kafka.connect.converters.ByteArrayConverter
target.dir = /backups
max.segment.size.bytes = 1073741824
cluster.bootstrap.servers = hostname-1.domain:9092,hostname-2.domain:9092,hostname-3.domain:9092
cluster.sasl.jaas.config = org.apache.kafka.common.security.scram.ScramLoginModule required username="REDACTED" password="REDACTED";
cluster.sasl_mechanism = SCRAM-SHA-256
cluster.security.protocol = SASL_SSL
cluster.sasl.mechanism = SCRAM-SHA-256
cluster.ssl.keystore.location = /path/to/ss/kafka.server.keystore.jks
cluster.ssl.keystore.password = REDACTED
cluster.ssl.key.password = REDACTED
cluster.ssl.truststore.location = /path/to/ss/kafka.server.truststore.jks
cluster.ssl.truststore.password = REDACTED

ํ•ด๋‹น ๋ผ์ธ์—์„œ jaas ๊ตฌ์„ฑ์„ ์ œ๊ฑฐํ•˜๊ณ  ๋‹ค๋ฅธ ๊ณณ์— ๋ฐฐ์น˜ํ•  ๊ฒƒ์ด์ง€๋งŒ ๋‹ค๋ฅธ ์‚ฌ๋žŒ๋“ค์„ ์˜ฌ๋ฐ”๋ฅธ ๊ธธ๋กœ ์ด๋„๋Š” ๋ฐ ๋„์›€์ด ๋˜๋„๋ก ๊ตฌ์„ฑ์„ ๊ณต์œ ํ•˜๋Š” ๊ฒƒ์ด ์˜ณ๋‹ค๊ณ  ์ƒ๊ฐํ–ˆ์Šต๋‹ˆ๋‹ค.

์˜ค๋Š˜ ๋ฐค v0.1์„ ์™„๋ฃŒํ•˜๊ธฐ ์œ„ํ•ด ๋‹ค์Œ ๋ฆด๋ฆฌ์Šค๋กœ ์ด๋™ํ•ฉ๋‹ˆ๋‹ค. ๊ทธ๋Ÿฌ๋‚˜ ์ด๊ฒƒ์€ ๋‚ด๊ฐ€ ๊ณง ํ•˜๊ณ  ์‹ถ์€ ์ผ์ž…๋‹ˆ๋‹ค.

์ด ํŽ˜์ด์ง€๊ฐ€ ๋„์›€์ด ๋˜์—ˆ๋‚˜์š”?
0 / 5 - 0 ๋“ฑ๊ธ‰