Kafka-backup: 无法反序列化主题上的标头“kafka_replyPartition”的值

创建于 2020-06-11  ·  15评论  ·  资料来源: itadventurer/kafka-backup

在我们的 QA 集群上出现另一个错误（kafka 2.4.0，kafka-backup 从提交 f30b9ad9 构建）。

[2020-06-11 08:39:55,585] WARN Failed to deserialize value for header 'kafka_replyPartition' on topic 'cosmos-cs-reads', so using byte array (org.apache.kafka.connect.storage.SimpleHeaderConverter:68)
java.lang.StringIndexOutOfBoundsException: String index out of range: 0
        at java.base/java.lang.StringLatin1.charAt(Unknown Source)
        at java.base/java.lang.String.charAt(Unknown Source)
        at org.apache.kafka.connect.data.Values.parse(Values.java:822)
        at org.apache.kafka.connect.data.Values.parseString(Values.java:378)
        at org.apache.kafka.connect.storage.SimpleHeaderConverter.toConnectHeader(SimpleHeaderConverter.java:64)
        at org.apache.kafka.connect.runtime.WorkerSinkTask.convertHeadersFor(WorkerSinkTask.java:516)
        at org.apache.kafka.connect.runtime.WorkerSinkTask.lambda$convertAndTransformRecord$3(WorkerSinkTask.java:491)
        at org.apache.kafka.connect.runtime.errors.RetryWithToleranceOperator.execAndRetry(RetryWithToleranceOperator.java:128)
        at org.apache.kafka.connect.runtime.errors.RetryWithToleranceOperator.execAndHandleError(RetryWithToleranceOperator.java:162)
        at org.apache.kafka.connect.runtime.errors.RetryWithToleranceOperator.execute(RetryWithToleranceOperator.java:104)
        at org.apache.kafka.connect.runtime.WorkerSinkTask.convertAndTransformRecord(WorkerSinkTask.java:491)
        at org.apache.kafka.connect.runtime.WorkerSinkTask.convertMessages(WorkerSinkTask.java:465)
        at org.apache.kafka.connect.runtime.WorkerSinkTask.poll(WorkerSinkTask.java:321)
        at org.apache.kafka.connect.runtime.WorkerSinkTask.iteration(WorkerSinkTask.java:224)
        at org.apache.kafka.connect.runtime.WorkerSinkTask.execute(WorkerSinkTask.java:192)
        at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:177)
        at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:227)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
        at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
        at java.base/java.lang.Thread.run(Unknown Source)
[2020-06-11 08:39:56,295] ERROR WorkerSinkTask{id=chrono_qa-backup-sink-0} Task threw an uncaught and unrecoverable exception. Task is being killed and will not recover until manually restarted. (org.apache.kafka.connect.runtime.WorkerSinkTask:559)
org.apache.kafka.connect.errors.DataException: cosmos-cs-reads error: Not a byte array! cosmos-cs-cmds
        at de.azapps.kafkabackup.common.AlreadyBytesConverter.fromConnectData(AlreadyBytesConverter.java:19)
        at de.azapps.kafkabackup.common.record.RecordSerde.write(RecordSerde.java:121)
        at de.azapps.kafkabackup.common.segment.SegmentWriter.append(SegmentWriter.java:75)
        at de.azapps.kafkabackup.common.partition.PartitionWriter.append(PartitionWriter.java:57)
        at de.azapps.kafkabackup.sink.BackupSinkTask.put(BackupSinkTask.java:68)
        at org.apache.kafka.connect.runtime.WorkerSinkTask.deliverMessages(WorkerSinkTask.java:539)
        at org.apache.kafka.connect.runtime.WorkerSinkTask.poll(WorkerSinkTask.java:322)
        at org.apache.kafka.connect.runtime.WorkerSinkTask.iteration(WorkerSinkTask.java:224)
        at org.apache.kafka.connect.runtime.WorkerSinkTask.execute(WorkerSinkTask.java:192)
        at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:177)
        at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:227)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
        at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
        at java.base/java.lang.Thread.run(Unknown Source)
[2020-06-11 08:39:56,353] ERROR WorkerSinkTask{id=chrono_qa-backup-sink-0} Task threw an uncaught and unrecoverable exception (org.apache.kafka.connect.runtime.WorkerTask:179)
org.apache.kafka.connect.errors.ConnectException: Exiting WorkerSinkTask due to unrecoverable exception.
        at org.apache.kafka.connect.runtime.WorkerSinkTask.deliverMessages(WorkerSinkTask.java:561)
        at org.apache.kafka.connect.runtime.WorkerSinkTask.poll(WorkerSinkTask.java:322)
        at org.apache.kafka.connect.runtime.WorkerSinkTask.iteration(WorkerSinkTask.java:224)
        at org.apache.kafka.connect.runtime.WorkerSinkTask.execute(WorkerSinkTask.java:192)
        at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:177)
        at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:227)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
        at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
        at java.base/java.lang.Thread.run(Unknown Source)
Caused by: org.apache.kafka.connect.errors.DataException: cosmos-cs-reads error: Not a byte array! cosmos-cs-cmds
        at de.azapps.kafkabackup.common.AlreadyBytesConverter.fromConnectData(AlreadyBytesConverter.java:19)
        at de.azapps.kafkabackup.common.record.RecordSerde.write(RecordSerde.java:121)
        at de.azapps.kafkabackup.common.segment.SegmentWriter.append(SegmentWriter.java:75)
        at de.azapps.kafkabackup.common.partition.PartitionWriter.append(PartitionWriter.java:57)
        at de.azapps.kafkabackup.sink.BackupSinkTask.put(BackupSinkTask.java:68)
        at org.apache.kafka.connect.runtime.WorkerSinkTask.deliverMessages(WorkerSinkTask.java:539)
        ... 10 more
[2020-06-11 08:39:56,354] ERROR WorkerSinkTask{id=chrono_qa-backup-sink-0} Task is being killed and will not recover until manually restarted (org.apache.kafka.connect.runtime.WorkerTask:180)
[2020-06-11 08:39:56,382] INFO Stopped BackupSinkTask (de.azapps.kafkabackup.sink.BackupSinkTask:139)
picture jay7x

所有15条评论

JFYI，这个话题对我来说很新。 我猜它是最近刚刚创建的。

jay7x picture jay7x 于 2020-06-11

JFYI 2，重新启动没有帮助。

jay7x picture jay7x 于 2020-06-11

以下是主题说明：

Topic: cosmos-cs-reads  PartitionCount: 1       ReplicationFactor: 1    Configs: compression.type=producer,min.insync.replicas=1,cleanup.policy=delete,segment.bytes=1073741824,flush.messages=9223372036854775807,file.delete.delay.ms=60000,max.message.bytes=1000012,min.compaction.lag.ms=0,message.timestamp.type=CreateTime,preallocate=false,index.interval.bytes=4096,min.cleanable.dirty.ratio=0.5,unclean.leader.election.enable=true,retention.bytes=120000000000,delete.retention.ms=86400000,message.timestamp.difference.max.ms=9223372036854775807,segment.index.bytes=10485760
        Topic: cosmos-cs-reads  Partition: 0    Leader: 1       Replicas: 1     Isr: 1
jay7x picture jay7x 于 2020-06-11

我尝试使用 kafka-console-consumer 消费消息，并且里面有有效的 JSON.. jq可以解析它没有问题。 虽然很少有字段是空字符串（ "field1":"","field2":"" ）。

jay7x picture jay7x 于 2020-06-11

好像标题有问题。 你能检查一下标题是什么样的吗？

itadventurer picture itadventurer 于 2020-06-14

你能建议怎么做吗？

jay7x picture jay7x 于 2020-06-14

最简单的方法是使用kafkacat： https ://stackoverflow.com/questions/55180620/how-to-view-kafka-headers

itadventurer picture itadventurer 于 2020-06-14

是的.. 刚试过kafkacat并且现在在标题中看到这个..
kafka_replyTopic=cosmos-cs-cmds,kafka_replyPartition=,kafka_correlationId=���D����� ;X,__TypeId__=eu.chargetime.ocpp.model.core.BootNotificationRequest

看来我们的开发人员做错了什么:(

UPD：他们告诉这是测试数据，这是预期的.. :(
UPD2：他们告诉kafka_*标头是由 Spring 云框架添加的。

jay7x picture jay7x 于 2020-06-15

好的，谢谢……我将不得不考虑如何解决这个问题。 我看这很关键😱

itadventurer picture itadventurer 于 2020-06-17
👍1

JFYI，我将其保持在“损坏”状态以测试您可能的修复方法：-D

jay7x picture jay7x 于 2020-06-18
👍1

嗯……我试图重现 #97 中的错误。 问题没有出现（在我的本地机器上 - GitHub Pipeline 正在运行）......

您可以尝试在此处添加有问题的标题消息吗？ https://github.com/itadventurer/kafka-backup/pull/97/files#diff -28c62e6ea255f4a9955c7be8c5d8a1cfR95
（显然是十六进制编码的数据）

希望我们能够在这里重现它;)

itadventurer picture itadventurer 于 2020-06-20
👍1

谢谢！ 我这周要试试！

jay7x picture jay7x 于 2020-06-21

我也无法重现它。 但是我意识到在提交 f30b9ad（这是我的构建基于）之后很少有提交涉及头文件处理。 所以我想我必须修改并在我的环境中重试。

jay7x picture jay7x 于 2020-07-02

升级和配置调整后，它工作正常......
我想主要原因是这个：

 connector.class=de.azapps.kafkabackup.sink.BackupSinkConnector
-key.converter=de.azapps.kafkabackup.common.AlreadyBytesConverter
-value.converter=de.azapps.kafkabackup.common.AlreadyBytesConverter
+key.converter=org.apache.kafka.connect.converters.ByteArrayConverter
+value.converter=org.apache.kafka.connect.converters.ByteArrayConverter
+header.converter=org.apache.kafka.connect.converters.ByteArrayConverter
jay7x picture jay7x 于 2020-07-03

关闭作为解决然后。

jay7x picture jay7x 于 2020-07-03
此页面是否有帮助？
0 / 5 - 0 等级

相关问题

jay7x picture jay7x  ·  18评论
itadventurer picture itadventurer  ·  5评论
ipochi picture ipochi  ·  3评论
akamensky picture akamensky  ·  13评论
huyngopt1994 picture huyngopt1994  ·  5评论