Kafka-backup: 如何使用正则表达式备份所有主题

创建于 2020-12-01  ·  5评论  ·  资料来源: itadventurer/kafka-backup

我已经使用“*”作为 topic_regex 来备份当前集群中的所有主题。 有什么帮助吗？ 谢谢@itadventurer

picture huyngopt1994

所有5条评论

嗨@huyngopt1994 ，你遇到了什么样的问题？ 与#116 有什么相似之处吗？

paoloyx picture paoloyx 于 2020-12-01
👍1

backup-standalone.sh --bootstrap-server broker:9092 --target-dir /kafka-backup --topics-regex .+

ogurskiy picture ogurskiy 于 2020-12-01
👍1

@ogurskiy谢谢。 那应该工作。

itadventurer picture itadventurer 于 2020-12-01
👍1

谢谢你，它有效，但在最新的主分支中运行备份时，我遇到了另一个永远循环的问题
命令运行
嗨@itadventurer

backup-standalone.sh --snapshot --bootstrap-server http://kafka-stg.default.svc.cluster.local:9092 --target-dir kafka-backup-2020-12-02-05-47 --topics sms_stgwechat_user --debug

[2020-12-02 07:38:50,469] DEBUG Flushed Topic sms_stgwechat_user, Partition 0 (de.azapps.kafkabackup.sink.BackupSinkTask)
[2020-12-02 07:39:00,506] DEBUG Flushed Topic sms_stgwechat_user, Partition 0 (de.azapps.kafkabackup.sink.BackupSinkTask)
[2020-12-02 07:39:10,546] DEBUG Flushed Topic sms_stgwechat_user, Partition 0 (de.azapps.kafkabackup.sink.BackupSinkTask)
[2020-12-02 07:39:20,581] DEBUG Flushed Topic sms_stgwechat_user, Partition 0 (de.azapps.kafkabackup.sink.BackupSinkTask)
[2020-12-02 07:39:30,617] DEBUG Flushed Topic sms_stgwechat_user, Partition 0 (de.azapps.kafkabackup.sink.BackupSinkTask)
[2020-12-02 07:39:40,669] DEBUG Flushed Topic sms_stgwechat_user, Partition 0 (de.azapps.kafkabackup.sink.BackupSinkTask)
[2020-12-02 07:39:50,709] DEBUG Flushed Topic sms_stgwechat_user, Partition 0 (de.azapps.kafkabackup.sink.BackupSinkTask)
[2020-12-02 07:40:00,765] DEBUG Flushed Topic sms_stgwechat_user, Partition 0 (de.azapps.kafkabackup.sink.BackupSinkTask)
[2020-12-02 07:40:10,806] DEBUG Flushed Topic sms_stgwechat_user, Partition 0 (de.azapps.kafkabackup.sink.BackupSinkTask)
[2020-12-02 07:40:20,842] DEBUG Flushed Topic sms_stgwechat_user, Partition 0 (de.azapps.kafkabackup.sink.BackupSinkTask)
[2020-12-02 07:40:30,896] DEBUG Flushed Topic sms_stgwechat_user, Partition 0 (de.azapps.kafkabackup.sink.BackupSinkTask)
[2020-12-02 07:40:40,934] DEBUG Flushed Topic sms_stgwechat_user, Partition 0 (de.azapps.kafkabackup.sink.BackupSinkTask)
[2020-12-02 07:40:50,972] DEBUG Flushed Topic sms_stgwechat_user, Partition 0 (de.azapps.kafkabackup.sink.BackupSinkTask)
[2020-12-02 07:41:01,012] DEBUG Flushed Topic sms_stgwechat_user, Partition 0 (de.azapps.kafkabackup.sink.BackupSinkTask)
[2020-12-02 07:41:11,053] DEBUG Flushed Topic sms_stgwechat_user, Partition 0 (de.azapps.kafkabackup.sink.BackupSinkTask)
[2020-12-02 07:41:21,094] DEBUG Flushed Topic sms_stgwechat_user, Partition 0 (de.azapps.kafkabackup.sink.BackupSinkTask)
[2020-12-02 07:41:31,135] DEBUG Flushed Topic sms_stgwechat_user, Partition 0 (de.azapps.kafkabackup.sink.BackupSinkTask)
[2020-12-02 07:41:41,168] DEBUG Flushed Topic sms_stgwechat_user, Partition 0 (de.azapps.kafkabackup.sink.BackupSinkTask)
[2020-12-02 07:41:51,207] DEBUG Flushed Topic sms_stgwechat_user, Partition 0 (de.azapps.kafkabackup.sink.BackupSinkTask)
[2020-12-02 07:42:01,261] DEBUG Flushed Topic sms_stgwechat_user, Partition 0 (de.azapps.kafkabackup.sink.BackupSinkTask)
[2020-12-02 07:42:11,296] DEBUG Flushed Topic sms_stgwechat_user, Partition 0 (de.azapps.kafkabackup.sink.BackupSinkTask)
[2020-12-02 07:42:21,337] DEBUG Flushed Topic sms_stgwechat_user, Partition 0 (de.azapps.kafkabackup.sink.BackupSinkTask)
[2020-12-02 07:42:31,378] DEBUG Flushed Topic sms_stgwechat_user, Partition 0 (de.azapps.kafkabackup.sink.BackupSinkTask)
[2020-12-02 07:42:41,416] DEBUG Flushed Topic sms_stgwechat_user, Partition 0 (de.azapps.kafkabackup.sink.BackupSinkTask)
[2020-12-02 07:42:51,454] DEBUG Flushed Topic sms_stgwechat_user, Partition 0 (de.azapps.kafkabackup.sink.BackupSinkTask)
[2020-12-02 07:43:01,496] DEBUG Flushed Topic sms_stgwechat_user, Partition 0 (de.azapps.kafkabackup.sink.BackupSinkTask)
[2020-12-02 07:43:11,536] DEBUG Flushed Topic sms_stgwechat_user, Partition 0 (de.azapps.kafkabackup.sink.BackupSinkTask)
[2020-12-02 07:43:21,582] DEBUG Flushed Topic sms_stgwechat_user, Partition 0 (de.azapps.kafkabackup.sink.BackupSinkTask)[2020-12-02 07:38:50,469] DEBUG Flushed Topic sms_stgwechat_user, Partition 0 (de.azapps.kafkabackup.sink.BackupSinkTask)
[2020-12-02 07:39:00,506] DEBUG Flushed Topic sms_stgwechat_user, Partition 0 (de.azapps.kafkabackup.sink.BackupSinkTask)
[2020-12-02 07:39:10,546] DEBUG Flushed Topic sms_stgwechat_user, Partition 0 (de.azapps.kafkabackup.sink.BackupSinkTask)
[2020-12-02 07:39:20,581] DEBUG Flushed Topic sms_stgwechat_user, Partition 0 (de.azapps.kafkabackup.sink.BackupSinkTask)
[2020-12-02 07:39:30,617] DEBUG Flushed Topic sms_stgwechat_user, Partition 0 (de.azapps.kafkabackup.sink.BackupSinkTask)
[2020-12-02 07:39:40,669] DEBUG Flushed Topic sms_stgwechat_user, Partition 0 (de.azapps.kafkabackup.sink.BackupSinkTask)
[2020-12-02 07:39:50,709] DEBUG Flushed Topic sms_stgwechat_user, Partition 0 (de.azapps.kafkabackup.sink.BackupSinkTask)
[2020-12-02 07:40:00,765] DEBUG Flushed Topic sms_stgwechat_user, Partition 0 (de.azapps.kafkabackup.sink.BackupSinkTask)
[2020-12-02 07:40:10,806] DEBUG Flushed Topic sms_stgwechat_user, Partition 0 (de.azapps.kafkabackup.sink.BackupSinkTask)
[2020-12-02 07:40:20,842] DEBUG Flushed Topic sms_stgwechat_user, Partition 0 (de.azapps.kafkabackup.sink.BackupSinkTask)
[2020-12-02 07:40:30,896] DEBUG Flushed Topic sms_stgwechat_user, Partition 0 (de.azapps.kafkabackup.sink.BackupSinkTask)
[2020-12-02 07:40:40,934] DEBUG Flushed Topic sms_stgwechat_user, Partition 0 (de.azapps.kafkabackup.sink.BackupSinkTask)
[2020-12-02 07:40:50,972] DEBUG Flushed Topic sms_stgwechat_user, Partition 0 (de.azapps.kafkabackup.sink.BackupSinkTask)
[2020-12-02 07:41:01,012] DEBUG Flushed Topic sms_stgwechat_user, Partition 0 (de.azapps.kafkabackup.sink.BackupSinkTask)
[2020-12-02 07:41:11,053] DEBUG Flushed Topic sms_stgwechat_user, Partition 0 (de.azapps.kafkabackup.sink.BackupSinkTask)
[2020-12-02 07:41:21,094] DEBUG Flushed Topic sms_stgwechat_user, Partition 0 (de.azapps.kafkabackup.sink.BackupSinkTask)
[2020-12-02 07:41:31,135] DEBUG Flushed Topic sms_stgwechat_user, Partition 0 (de.azapps.kafkabackup.sink.BackupSinkTask)
[2020-12-02 07:41:41,168] DEBUG Flushed Topic sms_stgwechat_user, Partition 0 (de.azapps.kafkabackup.sink.BackupSinkTask)
[2020-12-02 07:41:51,207] DEBUG Flushed Topic sms_stgwechat_user, Partition 0 (de.azapps.kafkabackup.sink.BackupSinkTask)
[2020-12-02 07:42:01,261] DEBUG Flushed Topic sms_stgwechat_user, Partition 0 (de.azapps.kafkabackup.sink.BackupSinkTask)
[2020-12-02 07:42:11,296] DEBUG Flushed Topic sms_stgwechat_user, Partition 0 (de.azapps.kafkabackup.sink.BackupSinkTask)
[2020-12-02 07:42:21,337] DEBUG Flushed Topic sms_stgwechat_user, Partition 0 (de.azapps.kafkabackup.sink.BackupSinkTask)
[2020-12-02 07:42:31,378] DEBUG Flushed Topic sms_stgwechat_user, Partition 0 (de.azapps.kafkabackup.sink.BackupSinkTask)
[2020-12-02 07:42:41,416] DEBUG Flushed Topic sms_stgwechat_user, Partition 0 (de.azapps.kafkabackup.sink.BackupSinkTask)
[2020-12-02 07:42:51,454] DEBUG Flushed Topic sms_stgwechat_user, Partition 0 (de.azapps.kafkabackup.sink.BackupSinkTask)
[2020-12-02 07:43:01,496] DEBUG Flushed Topic sms_stgwechat_user, Partition 0 (de.azapps.kafkabackup.sink.BackupSinkTask)
[2020-12-02 07:43:11,536] DEBUG Flushed Topic sms_stgwechat_user, Partition 0 (de.azapps.kafkabackup.sink.BackupSinkTask)
[2020-12-02 07:43:21,582] DEBUG Flushed Topic sms_stgwechat_user, Partition 0 (de.azapps.kafkabackup.sink.BackupSinkTask)

好像消费者不能消费消息

Screen Shot 2020-12-02 at 2 46 42 PM

kafka-topics --zookeeper kafka-stg-zookeeper:2181 --topic sms_stgwechat_user --describe
Topic:sms_stgwechat_user    PartitionCount:1    ReplicationFactor:1 Configs:
    Topic: sms_stgwechat_user   Partition: 0    Leader: 2   Replicas: 2 Isr: 2

任何提示谢谢大家

huyngopt1994 picture huyngopt1994 于 2020-12-02

嗨@huyngopt1994 ，你遇到了什么样的问题？ 与#116 有什么相似之处吗？

啊不，不一样

huyngopt1994 picture huyngopt1994 于 2020-12-02
此页面是否有帮助？
0 / 5 - 0 等级

相关问题

jay7x picture jay7x  ·  15评论
jay7x picture jay7x  ·  16评论
itadventurer picture itadventurer  ·  5评论
jay7x picture jay7x  ·  18评论
akamensky picture akamensky  ·  13评论