描述错误
EventHubProducerClient 停止发送批处理并在异常中给出不同的maximumSizeInBytes
应该是什么。
异常或堆栈跟踪
20:33:45 [main] WARN c.a.c.a.i.RetryUtil - Error is not a TimeoutException nor is it a retryable AMQP exception.
Size of the payload exceeded maximum message size: 256 kb, errorContext[NAMESPACE: mmp-streams.servicebus.windows.net, PATH: billing, REFERENCE_ID: 6d0df19120c4489a8f405b86a7b5ded1_G31, LINK_CREDIT: 300]
20:33:45 [main] ERROR a.e.ProtoProducer - Exception:
com.azure.core.amqp.exception.AmqpException: Size of the payload exceeded maximum message size: 256 kb, errorContext[NAMESPACE: mmp-streams.servicebus.windows.net, PATH: billing, REFERENCE_ID: 6d0df19120c4489a8f405b86a7b5ded1_G31, LINK_CREDIT: 300]
at com.azure.core.amqp.implementation.ReactorSender.send(ReactorSender.java:217)
at com.azure.messaging.eventhubs.EventHubProducerAsyncClient.lambda$send$6(EventHubProducerAsyncClient.java:437)
at reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:118)
at reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1705)
at reactor.core.publisher.MonoFlatMap$FlatMapInner.onNext(MonoFlatMap.java:241)
at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onNext(FluxMapFuseable.java:121)
at reactor.core.publisher.Operators$ScalarSubscription.request(Operators.java:2267)
at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.request(FluxMapFuseable.java:162)
at reactor.core.publisher.MonoFlatMap$FlatMapInner.onSubscribe(MonoFlatMap.java:230)
at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onSubscribe(FluxMapFuseable.java:90)
at reactor.core.publisher.MonoJust.subscribe(MonoJust.java:54)
at reactor.core.publisher.Mono.subscribe(Mono.java:4110)
at reactor.core.publisher.FluxFlatMap.trySubscribeScalarMap(FluxFlatMap.java:199)
at reactor.core.publisher.MonoFlatMap.subscribeOrReturn(MonoFlatMap.java:53)
at reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:48)
at reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:150)
at reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1705)
at com.azure.core.amqp.implementation.AmqpChannelProcessor.subscribe(AmqpChannelProcessor.java:220)
at reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:55)
at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:52)
at reactor.core.publisher.FluxRetryWhen.subscribe(FluxRetryWhen.java:87)
at reactor.core.publisher.MonoRetryWhen.subscribeOrReturn(MonoRetryWhen.java:50)
at reactor.core.publisher.Mono.subscribe(Mono.java:4095)
at reactor.core.publisher.Mono.block(Mono.java:1663)
at com.azure.messaging.eventhubs.EventHubProducerClient.send(EventHubProducerClient.java:218)
at app.eh.ProtoProducer.publish(ProtoProducer.java:67)
at app.workers.Processor.lambda$deductFunds$6(Processor.java:216)
at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(Unknown Source)
at java.base/java.util.stream.ReferencePipeline$Head.forEach(Unknown Source)
at app.workers.Processor.deductFunds(Processor.java:208)
at app.workers.Processor.process(Processor.java:263)
at app.Main.main(Main.java:43)
Suppressed: java.lang.Exception: #block terminated with an error
at reactor.core.publisher.BlockingSingleSubscriber.blockingGet(BlockingSingleSubscriber.java:99)
at reactor.core.publisher.Mono.block(Mono.java:1664)
... 8 common frames omitted
Caused by: java.nio.BufferOverflowException: null
at org.apache.qpid.proton.codec.WritableBuffer$ByteBufferWrapper.ensureRemaining(WritableBuffer.java:152)
at org.apache.qpid.proton.codec.BinaryType.fastWrite(BinaryType.java:83)
at org.apache.qpid.proton.codec.EncoderImpl.writeBinary(EncoderImpl.java:535)
at org.apache.qpid.proton.codec.messaging.FastPathDataType.write(FastPathDataType.java:126)
at org.apache.qpid.proton.codec.messaging.FastPathDataType.write(FastPathDataType.java:36)
at org.apache.qpid.proton.codec.EncoderImpl.writeObject(EncoderImpl.java:734)
at org.apache.qpid.proton.message.impl.MessageImpl.encode(MessageImpl.java:740)
at org.apache.qpid.proton.message.impl.MessageImpl.encode(MessageImpl.java:696)
at com.azure.core.amqp.implementation.ReactorSender.send(ReactorSender.java:210)
... 31 common frames omitted
Caused by: java.lang.IndexOutOfBoundsException: Requested min remaining bytes(437) exceeds remaining(314) in underlying ByteBuffer: java.nio.HeapByteBuffer[pos=261829 lim=262143 cap=262144]
at org.apache.qpid.proton.codec.WritableBuffer$ByteBufferWrapper.ensureRemaining(WritableBuffer.java:148)
... 39 common frames omitted
再现
重现行为的步骤:
开始发送没有提供maximumSizeInBytes
值的消息(库自动将其填充为1022
KB)并模拟低流量,其中需要 30-35 分钟来填充一个批次。 使用的代码与此处显示的相同: https : 5.0.3
代码片段
try {
if (!eventDataBatch.tryAdd(new EventData(message.toByteArray()))) {
eventHubProducerClient.send(eventDataBatch);
eventDataBatch = eventHubProducerClient.createBatch();
// Try to add that event that couldn't fit before.
if (!eventDataBatch.tryAdd(new EventData(message.toByteArray()))) {
log.error("Event is too large for an empty batch. Max size: " + eventDataBatch.getMaxSizeInBytes());
}
}
} catch (Exception e) {
eventDataBatch = eventHubProducerClient.createBatch();
log.error("Exception: ", e);
}
预期行为
如果消息太大,则应首先将已创建的批次发送到 EventHub,而不是异常,然后应创建一个新批次,并在其中插入消息。 相反,正在抛出消息有效负载大小大异常。
附加上下文
这也只发生在低流量时,需要很长时间才能填满一个批次(默认大小为 1022KB)。 这在同一时间范围内发生了两次,其中流量较低,连续eventHubProducerClient.send(eventDataBatch)
有> 30
分钟的时间间隔。
还要指出的是,当我没有以字节maximumBatchSize
单位给出1022
KB 并且突然在堆栈跟踪中,大小被赋予是256
KB 而不是1022
KB。
据我所知, maximumLinkSize
被视为maximumSizeInBytes
。 如果链接断开,它会回退到256
KB 并且由于已经创建的批量大小大于256
KB,我们会收到此错误吗?
我从EventHubProducerAsyncClient
类的以下方法中的这个方法得到了这个:
public Mono<EventDataBatch> createBatch(CreateBatchOptions options) {
if (options == null) {
return FluxUtil.monoError(this.logger, new NullPointerException("'options' cannot be null."));
} else {
String partitionKey = options.getPartitionKey();
String partitionId = options.getPartitionId();
int batchMaxSize = options.getMaximumSizeInBytes();
if (!CoreUtils.isNullOrEmpty(partitionKey) && !CoreUtils.isNullOrEmpty(partitionId)) {
return FluxUtil.monoError(this.logger, new IllegalArgumentException(String.format(Locale.US, "CreateBatchOptions.getPartitionKey() and CreateBatchOptions.getPartitionId() are both set. Only one or the other can be used. partitionKey: '%s'. partitionId: '%s'", partitionKey, partitionId)));
} else {
return !CoreUtils.isNullOrEmpty(partitionKey) && partitionKey.length() > 128 ? FluxUtil.monoError(this.logger, new IllegalArgumentException(String.format(Locale.US, "Partition key '%s' exceeds the maximum allowed length: '%s'.", partitionKey, 128))) : this.getSendLink(partitionId).flatMap((link) -> {
return link.getLinkSize().flatMap((size) -> {
int maximumLinkSize = size > 0 ? size : 262144;
if (batchMaxSize > maximumLinkSize) {
return FluxUtil.monoError(this.logger, new IllegalArgumentException(String.format(Locale.US, "BatchOptions.maximumSizeInBytes (%s bytes) is larger than the link size (%s bytes).", batchMaxSize, maximumLinkSize)));
} else {
int batchSize = batchMaxSize > 0 ? batchMaxSize : maximumLinkSize;
Objects.requireNonNull(link);
return Mono.just(new EventDataBatch(batchSize, partitionId, partitionKey, link::getErrorContext, this.tracerProvider, link.getEntityPath(), link.getHostname()));
}
});
});
}
}
}
库版本是:
<dependency>
<groupId>com.azure</groupId>
<artifactId>azure-messaging-eventhubs</artifactId>
<version>5.0.3</version>
</dependency>
信息清单
请确保您已添加以上所有以下信息并勾选必填字段,否则我们会将发行人视为不完整的报告
如果在AmqpException
发生后重新制作EventHubProducerClient
我可以缓解这个问题。
以下是我正在使用的代码:
// tobePushed is -> List<Message> toBePushed = new ArrayList<>();
public void publish(Message message) {
toBePushed.add(billing);
try {
if (!eventDataBatch.tryAdd(new EventData(message.toByteArray()))) {
eventHubProducerClient.send(eventDataBatch);
toBePushed = new ArrayList<>();
markBatchMetrics(eventDataBatch);
eventDataBatch = eventHubProducerClient.createBatch();
// Try to add that event that couldn't fit before.
if (!eventDataBatch.tryAdd(new EventData(message.toByteArray()))) {
log.error("Event is too large for an empty batch. Max size: " + eventDataBatch.getMaxSizeInBytes());
}
}
} catch (AmqpException e) {
log.error("AmqpException: ", e);
eventHubProducerClient = new EventHubClientBuilder()
.connectionString(connectionString)
.buildProducerClient();
eventDataBatch = eventHubProducerClient.createBatch();
for (Message message1: messages) {
if (!eventDataBatch.tryAdd(new EventData(message1.toByteArray()))) {
eventHubProducerClient.send(eventDataBatch);
toBePushed = new ArrayList<>();
eventDataBatch = eventHubProducerClient.createBatch();
// Try to add that event that couldn't fit before.
if (!eventDataBatch.tryAdd(new EventData(message1.toByteArray()))) {
log.error("Event is too large for an empty batch. Max size: " + eventDataBatch.getMaxSizeInBytes());
}
}
}
eventHubProducerClient.send(eventDataBatch);
eventDataBatch = eventHubProducerClient.createBatch();
} catch (Exception e) {
eventDataBatch = eventHubProducerClient.createBatch();
log.error("Exception: ", e);
}
是否有一个配置可以让我控制EventHubProducerClient
保持连接的时间?
感谢@shubhambhattar 报告此问题。 @srnagar你能调查一下吗?
/cc @康尼
我创建了一个小的示例代码来重现它。 上面分享的代码并不完整,无法原样运行。 这个可以运行,因为它只是包含依赖项(版本5.0.3
)。
import com.azure.messaging.eventhubs.EventData;
import com.azure.messaging.eventhubs.EventDataBatch;
import com.azure.messaging.eventhubs.EventHubClientBuilder;
import com.azure.messaging.eventhubs.EventHubProducerClient;
import java.time.Instant;
public class ProducerTest {
private static EventHubProducerClient eventHubProducerClient;
private static EventDataBatch batch;
private static void addToBatch() {
try {
for(int i = 0; i < 45000; i++)
batch.tryAdd(new EventData("test".getBytes()));
} catch (Exception e) {
System.out.println(e.getMessage());
}
}
private static void send(EventDataBatch batch) {
try {
eventHubProducerClient.send(batch);
} catch (Exception e) {
System.out.println(e.getMessage());
}
}
public static void main(String[] args) {
eventHubProducerClient = new EventHubClientBuilder()
.connectionString("saskey")
.buildProducerClient();
batch = eventHubProducerClient.createBatch();
addToBatch();
System.out.println("MaxSize = " + batch.getMaxSizeInBytes() / 1024.0d + " KB, msgSize = " + batch.getSizeInBytes() / 1024.0d + " KB");
send(batch);
long timePushed = Instant.now().getEpochSecond();
batch = eventHubProducerClient.createBatch();
addToBatch();
while(true) {
// wait till 31 mins
if (Instant.now().getEpochSecond() - timePushed > 31 * 60) {
System.out.println("MaxSize = " + batch.getMaxSizeInBytes() / 1024.0d + " KB, msgSize = " + batch.getSizeInBytes() / 1024.0d + " KB");
send(batch);
break;
}
}
}
}
对应的日志是:
19:50:43 [main] INFO c.a.m.e.EventHubClientBuilder - connectionId[MF_fc8b42_1588602043945]: Emitting a single connection.
19:50:43 [main] INFO c.a.m.e.i.EventHubConnectionProcessor - connectionId[some-namespace.servicebus.windows.net] entityPath[dummy]: Setting next AMQP channel.
19:50:43 [main] INFO c.a.c.a.i.ReactorConnection - connectionId[MF_fc8b42_1588602043945]: Creating and starting connection to some-namespace.servicebus.windows.net:5671
19:50:44 [main] INFO c.a.c.a.i.ReactorExecutor - connectionId[MF_fc8b42_1588602043945], message[Starting reactor.]
19:50:44 [single-1] INFO c.a.c.a.i.h.ConnectionHandler - onConnectionInit hostname[some-namespace.servicebus.windows.net], connectionId[MF_fc8b42_1588602043945]
19:50:44 [single-1] INFO c.a.c.a.i.h.ReactorHandler - connectionId[MF_fc8b42_1588602043945] reactor.onReactorInit
19:50:44 [main] INFO c.a.c.a.i.AzureTokenManagerProvider - Creating new token manager for audience[amqp://some-namespace.servicebus.windows.net/dummy], resource[dummy]
19:50:44 [single-1] INFO c.a.c.a.i.h.ConnectionHandler - onConnectionLocalOpen hostname[some-namespace.servicebus.windows.net:5671], connectionId[MF_fc8b42_1588602043945], errorCondition[null], errorDescription[null]
19:50:44 [single-1] INFO c.a.c.a.i.h.ConnectionHandler - onConnectionBound hostname[some-namespace.servicebus.windows.net], connectionId[MF_fc8b42_1588602043945]
19:50:47 [single-1] INFO c.a.c.a.i.h.ConnectionHandler - onConnectionRemoteOpen hostname[some-namespace.servicebus.windows.net:5671], connectionId[MF_fc8b42_1588602043945], remoteContainer[68897ad4abc542519d69b6e61a1579ba_G18]
19:50:47 [single-1] INFO c.a.c.a.i.h.SessionHandler - onSessionRemoteOpen connectionId[MF_fc8b42_1588602043945], entityName[dummy], sessionIncCapacity[0], sessionOutgoingWindow[2147483647]
19:50:47 [single-1] INFO c.a.c.a.i.ReactorConnection - Setting CBS channel.
19:50:47 [single-1] INFO c.a.c.a.i.ReactorConnection - Emitting new response channel. connectionId: MF_fc8b42_1588602043945. entityPath: $cbs. linkName: cbs.
19:50:47 [single-1] INFO c.a.c.a.i.RequestResponseChannel<cbs-session> - connectionId[MF_fc8b42_1588602043945] entityPath[$cbs]: Setting next AMQP channel.
19:50:47 [single-1] INFO c.a.c.a.i.h.ReceiveLinkHandler - onLinkLocalOpen connectionId[MF_fc8b42_1588602043945], linkName[cbs:receiver], localSource[Source{address='$cbs', durable=NONE, expiryPolicy=SESSION_END, timeout=0, dynamic=false, dynamicNodeProperties=null, distributionMode=null, filter=null, defaultOutcome=null, outcomes=null, capabilities=null}]
19:50:47 [single-1] INFO c.a.c.a.i.h.SessionHandler - onSessionRemoteOpen connectionId[MF_fc8b42_1588602043945], entityName[cbs-session], sessionIncCapacity[0], sessionOutgoingWindow[2147483647]
19:50:47 [single-1] INFO c.a.c.a.i.h.SendLinkHandler - onLinkRemoteOpen connectionId[MF_fc8b42_1588602043945], linkName[cbs:sender], remoteTarget[Target{address='$cbs', durable=NONE, expiryPolicy=SESSION_END, timeout=0, dynamic=false, dynamicNodeProperties=null, capabilities=null}]
19:50:47 [single-1] INFO c.a.c.a.i.h.ReceiveLinkHandler - onLinkRemoteOpen connectionId[MF_fc8b42_1588602043945], linkName[cbs:receiver], remoteSource[Source{address='$cbs', durable=NONE, expiryPolicy=SESSION_END, timeout=0, dynamic=false, dynamicNodeProperties=null, distributionMode=null, filter=null, defaultOutcome=null, outcomes=null, capabilities=null}]
19:50:48 [single-1] INFO c.a.c.a.i.ActiveClientTokenManager - Scheduling refresh token task. scopes[amqp://some-namespace.servicebus.windows.net/dummy]
19:50:49 [single-1] INFO c.a.c.a.i.h.SendLinkHandler - onLinkRemoteOpen connectionId[MF_fc8b42_1588602043945], linkName[dummy], remoteTarget[Target{address='dummy', durable=NONE, expiryPolicy=SESSION_END, timeout=0, dynamic=false, dynamicNodeProperties=null, capabilities=null}]
MaxSize = 1022.0 KB, msgSize = 1021.982421875 KB
MaxSize = 1022.0 KB, msgSize = 1021.982421875 KB
20:06:58 [parallel-3] INFO c.a.c.a.i.ActiveClientTokenManager - Refreshing token. scopes[amqp://some-namespace.servicebus.windows.net/dummy]
20:06:59 [single-1] INFO c.a.c.a.i.ActiveClientTokenManager - Authorization successful. Refreshing token in 971000 ms. scopes[amqp://some-namespace.servicebus.windows.net/dummy]
20:21:05 [single-1] INFO c.a.c.a.i.h.SendLinkHandler - onLinkRemoteClose connectionId[MF_fc8b42_1588602043945], linkName[dummy], errorCondition[amqp:link:detach-forced], errorDescription[Idle link tracker, link dummy has been idle for 1800000ms TrackingId:4baeefe9-6ae0-4635-8d4c-d963c589cb41_G18, SystemTracker:client-link13935138, Timestamp:2020-05-04T14:51:05]
20:21:05 [single-1] INFO c.a.c.a.i.h.SendLinkHandler - processOnClose connectionId[MF_fc8b42_1588602043945], linkName[dummy], errorCondition[amqp:link:detach-forced], errorDescription[Idle link tracker, link dummy has been idle for 1800000ms TrackingId:4baeefe9-6ae0-4635-8d4c-d963c589cb41_G18, SystemTracker:client-link13935138, Timestamp:2020-05-04T14:51:05]
20:21:05 [single-1] ERROR c.a.c.a.i.ReactorSender - [dummy] Error occurred in sender error handler.
Idle link tracker, link dummy has been idle for 1800000ms TrackingId:4baeefe9-6ae0-4635-8d4c-d963c589cb41_G18, SystemTracker:client-link13935138, Timestamp:2020-05-04T14:51:05, errorContext[NAMESPACE: some-namespace.servicebus.windows.net, PATH: dummy, REFERENCE_ID: 68897ad4abc542519d69b6e61a1579ba_G18, LINK_CREDIT: 299]
20:21:05 [single-1] INFO c.a.c.a.i.ReactorSession - linkName[dummy]: Error occurred. Removing and disposing send link.
20:21:05 [single-1] INFO c.a.c.a.i.h.SendLinkHandler - onLinkLocalClose connectionId[MF_fc8b42_1588602043945], linkName[dummy], errorCondition[amqp:link:detach-forced], errorDescription[Idle link tracker, link dummy has been idle for 1800000ms TrackingId:4baeefe9-6ae0-4635-8d4c-d963c589cb41_G18, SystemTracker:client-link13935138, Timestamp:2020-05-04T14:51:05]
20:26:05 [single-1] INFO c.a.c.a.i.h.SessionHandler - onSessionRemoteClose connectionId[dummy], entityName[MF_fc8b42_1588602043945], condition[Error{condition=null, description='null', info=null}]
20:26:05 [single-1] INFO c.a.c.a.i.h.SessionHandler - onSessionRemoteClose closing a local session for connectionId[MF_fc8b42_1588602043945], entityName[dummy], condition[null], description[null]
20:26:05 [single-1] INFO c.a.c.a.i.ReactorConnection - connectionId[MF_fc8b42_1588602043945] sessionName[dummy]: Complete. Removing and disposing session.
20:26:05 [single-1] INFO c.a.c.a.i.ReactorSession - sessionId[dummy]: Disposing of session.
20:26:05 [single-1] INFO c.a.c.a.i.h.SendLinkHandler - onLinkRemoteClose connectionId[MF_fc8b42_1588602043945], linkName[cbs:sender], errorCondition[amqp:connection:forced], errorDescription[The connection was inactive for more than the allowed 300000 milliseconds and is closed by container 'LinkTracker'. TrackingId:68897ad4abc542519d69b6e61a1579ba_G18, SystemTracker:gateway5, Timestamp:2020-05-04T14:56:05]
20:26:05 [single-1] INFO c.a.c.a.i.h.SendLinkHandler - processOnClose connectionId[MF_fc8b42_1588602043945], linkName[cbs:sender], errorCondition[amqp:connection:forced], errorDescription[The connection was inactive for more than the allowed 300000 milliseconds and is closed by container 'LinkTracker'. TrackingId:68897ad4abc542519d69b6e61a1579ba_G18, SystemTracker:gateway5, Timestamp:2020-05-04T14:56:05]
20:26:05 [single-1] WARN c.a.c.a.i.RequestResponseChannel<cbs-session> - Retry #1. Transient error occurred. Retrying after 4511 ms.
The connection was inactive for more than the allowed 300000 milliseconds and is closed by container 'LinkTracker'. TrackingId:68897ad4abc542519d69b6e61a1579ba_G18, SystemTracker:gateway5, Timestamp:2020-05-04T14:56:05, errorContext[NAMESPACE: some-namespace.servicebus.windows.net, PATH: $cbs, REFERENCE_ID: cbs:sender, LINK_CREDIT: 98]
20:26:05 [single-1] INFO c.a.c.a.i.h.ReceiveLinkHandler - onLinkRemoteClose connectionId[MF_fc8b42_1588602043945], linkName[cbs:receiver], errorCondition[amqp:connection:forced], errorDescription[The connection was inactive for more than the allowed 300000 milliseconds and is closed by container 'LinkTracker'. TrackingId:68897ad4abc542519d69b6e61a1579ba_G18, SystemTracker:gateway5, Timestamp:2020-05-04T14:56:05]
20:26:05 [single-1] INFO c.a.c.a.i.h.ReceiveLinkHandler - processOnClose connectionId[MF_fc8b42_1588602043945], linkName[cbs:receiver], errorCondition[amqp:connection:forced], errorDescription[The connection was inactive for more than the allowed 300000 milliseconds and is closed by container 'LinkTracker'. TrackingId:68897ad4abc542519d69b6e61a1579ba_G18, SystemTracker:gateway5, Timestamp:2020-05-04T14:56:05]
20:26:05 [single-1] INFO c.a.c.a.i.h.SessionHandler - onSessionRemoteClose connectionId[cbs-session], entityName[MF_fc8b42_1588602043945], condition[Error{condition=null, description='null', info=null}]
20:26:05 [single-1] INFO c.a.c.a.i.h.SessionHandler - onSessionRemoteClose closing a local session for connectionId[MF_fc8b42_1588602043945], entityName[cbs-session], condition[null], description[null]
20:26:05 [single-1] INFO c.a.c.a.i.ReactorConnection - connectionId[MF_fc8b42_1588602043945] sessionName[cbs-session]: Complete. Removing and disposing session.
20:26:05 [single-1] INFO c.a.c.a.i.ReactorSession - sessionId[cbs-session]: Disposing of session.
20:26:05 [single-1] INFO c.a.c.a.i.h.ConnectionHandler - onConnectionRemoteClose hostname[some-namespace.servicebus.windows.net:5671], connectionId[MF_fc8b42_1588602043945], errorCondition[amqp:connection:forced], errorDescription[The connection was inactive for more than the allowed 300000 milliseconds and is closed by container 'LinkTracker'. TrackingId:68897ad4abc542519d69b6e61a1579ba_G18, SystemTracker:gateway5, Timestamp:2020-05-04T14:56:05]
20:26:05 [single-1] INFO c.a.m.e.i.EventHubConnectionProcessor - Channel closed.
20:26:05 [single-1] INFO c.a.m.e.i.EventHubReactorAmqpConnection - connectionId[MF_fc8b42_1588602043945]: Disposing of connection.
20:26:05 [single-1] INFO c.a.c.a.i.ReactorConnection - connectionId[MF_fc8b42_1588602043945]: Disposing of ReactorConnection.
20:26:05 [single-1] INFO c.a.c.a.i.AmqpExceptionHandler - Shutdown received: ReactorExecutor.close() was called., isTransient[false], initiatedByClient[true]
20:26:10 [parallel-2] INFO c.a.c.a.i.RequestResponseChannel<cbs-session> - Retry #1. Requesting from upstream.
20:26:10 [parallel-2] INFO c.a.c.a.i.RequestResponseChannel<cbs-session> - connectionId[MF_fc8b42_1588602043945] entityPath[$cbs]: Connection not requested, yet. Requesting one.
20:26:10 [parallel-2] ERROR c.a.c.a.i.ReactorConnection - connectionId[MF_fc8b42_1588602043945]: Connection is disposed. Cannot get CBS node.
20:26:10 [parallel-2] INFO c.a.c.a.i.ReactorConnection - Emitting new response channel. connectionId: MF_fc8b42_1588602043945. entityPath: $cbs. linkName: cbs.
20:26:10 [parallel-2] INFO c.a.c.a.i.RequestResponseChannel<cbs-session> - connectionId[MF_fc8b42_1588602043945] entityPath[$cbs]: Setting next AMQP channel.
20:27:05 [single-1] INFO c.a.c.a.i.ReactorExecutor - Unable to acquire dispose reactor semaphore within timeout.
20:27:05 [single-1] INFO c.a.c.a.i.h.SendLinkHandler - onLinkLocalClose connectionId[MF_fc8b42_1588602043945], linkName[cbs:sender], errorCondition[amqp:connection:forced], errorDescription[The connection was inactive for more than the allowed 300000 milliseconds and is closed by container 'LinkTracker'. TrackingId:68897ad4abc542519d69b6e61a1579ba_G18, SystemTracker:gateway5, Timestamp:2020-05-04T14:56:05]
20:27:05 [single-1] INFO c.a.c.a.i.h.ReceiveLinkHandler - onLinkLocalClose connectionId[MF_fc8b42_1588602043945], linkName[cbs:receiver], errorCondition[null], errorDescription[null]
20:27:05 [single-1] INFO c.a.c.a.i.h.ConnectionHandler - onConnectionLocalClose hostname[some-namespace.servicebus.windows.net:5671], connectionId[MF_fc8b42_1588602043945], errorCondition[null], errorDescription[null]
20:27:05 [single-1] INFO c.a.c.a.i.h.ConnectionHandler - onConnectionUnbound hostname[some-namespace.servicebus.windows.net:5671], connectionId[MF_fc8b42_1588602043945], state[CLOSED], remoteState[CLOSED]
20:27:05 [single-1] INFO c.a.c.a.i.h.SendLinkHandler - onLinkFinal connectionId[MF_fc8b42_1588602043945], linkName[dummy]
20:27:05 [single-1] INFO c.a.c.a.i.h.SessionHandler - onSessionFinal connectionId[MF_fc8b42_1588602043945], entityName[dummy], condition[null], description[null]
20:27:05 [single-1] INFO c.a.c.a.i.h.SendLinkHandler - onLinkFinal connectionId[MF_fc8b42_1588602043945], linkName[cbs:sender]
20:27:05 [single-1] INFO c.a.c.a.i.h.ReceiveLinkHandler - onLinkFinal connectionId[MF_fc8b42_1588602043945], linkName[cbs:receiver]
20:27:05 [single-1] INFO c.a.c.a.i.h.SessionHandler - onSessionFinal connectionId[MF_fc8b42_1588602043945], entityName[cbs-session], condition[null], description[null]
20:27:05 [single-1] INFO c.a.c.a.i.h.SendLinkHandler - onLinkFinal connectionId[MF_fc8b42_1588602043945], linkName[cbs:sender]
20:27:05 [single-1] INFO c.a.c.a.i.h.ReceiveLinkHandler - onLinkFinal connectionId[MF_fc8b42_1588602043945], linkName[cbs:receiver]
20:27:05 [single-1] INFO c.a.c.a.i.h.SessionHandler - onSessionFinal connectionId[MF_fc8b42_1588602043945], entityName[cbs-session], condition[null], description[null]
20:27:05 [single-1] INFO c.a.c.a.i.ReactorConnection - connectionId[MF_fc8b42_1588602043945] sessionName[cbs-session]: Complete. Removing and disposing session.
20:27:05 [single-1] INFO c.a.c.a.i.ReactorSession - sessionId[cbs-session]: Disposing of session.
20:27:05 [single-1] INFO c.a.c.a.i.h.ConnectionHandler - onConnectionFinal hostname[some-namespace.servicebus.windows.net:5671], connectionId[MF_fc8b42_1588602043945], errorCondition[null], errorDescription[null]
20:27:05 [single-1] INFO c.a.c.a.i.h.SendLinkHandler - onLinkLocalClose connectionId[MF_fc8b42_1588602043945], linkName[cbs:sender], errorCondition[null], errorDescription[null]
20:27:05 [single-1] INFO c.a.c.a.i.h.ReceiveLinkHandler - onLinkLocalClose connectionId[MF_fc8b42_1588602043945], linkName[cbs:receiver], errorCondition[null], errorDescription[null]
20:27:05 [single-1] INFO c.a.c.a.i.ReactorExecutor - connectionId[MF_fc8b42_1588602043945], message[Processing all pending tasks and closing old reactor.]
20:27:05 [single-1] INFO c.a.c.a.i.h.ReceiveLinkHandler - onLinkLocalOpen connectionId[MF_fc8b42_1588602043945], linkName[cbs:receiver], localSource[Source{address='$cbs', durable=NONE, expiryPolicy=SESSION_END, timeout=0, dynamic=false, dynamicNodeProperties=null, distributionMode=null, filter=null, defaultOutcome=null, outcomes=null, capabilities=null}]
20:27:05 [single-1] INFO c.a.c.a.i.ReactorExecutor - connectionId[MF_fc8b42_1588602043945], message[Stopping the reactor because thread was interrupted or the reactor has no more events to process.]
20:31:11 [main] INFO c.a.m.e.i.EventHubConnectionProcessor - connectionId[some-namespace.servicebus.windows.net] entityPath[dummy]: Connection not requested, yet. Requesting one.
20:31:11 [main] INFO c.a.m.e.EventHubClientBuilder - connectionId[MF_65d28d_1588604471389]: Emitting a single connection.
20:31:11 [main] INFO c.a.m.e.i.EventHubConnectionProcessor - connectionId[some-namespace.servicebus.windows.net] entityPath[dummy]: Setting next AMQP channel.
20:31:11 [main] INFO c.a.c.a.i.ReactorConnection - connectionId[MF_65d28d_1588604471389]: Creating and starting connection to some-namespace.servicebus.windows.net:5671
20:31:11 [main] INFO c.a.c.a.i.ReactorExecutor - connectionId[MF_65d28d_1588604471389], message[Starting reactor.]
20:31:11 [single-1] INFO c.a.c.a.i.h.ConnectionHandler - onConnectionInit hostname[some-namespace.servicebus.windows.net], connectionId[MF_65d28d_1588604471389]
20:31:11 [main] INFO c.a.c.a.i.AzureTokenManagerProvider - Creating new token manager for audience[amqp://some-namespace.servicebus.windows.net/dummy], resource[dummy]
20:31:11 [single-1] INFO c.a.c.a.i.h.ReactorHandler - connectionId[MF_65d28d_1588604471389] reactor.onReactorInit
20:31:11 [single-1] INFO c.a.c.a.i.h.ConnectionHandler - onConnectionLocalOpen hostname[some-namespace.servicebus.windows.net:5671], connectionId[MF_65d28d_1588604471389], errorCondition[null], errorDescription[null]
20:31:11 [single-1] INFO c.a.c.a.i.h.ConnectionHandler - onConnectionBound hostname[some-namespace.servicebus.windows.net], connectionId[MF_65d28d_1588604471389]
20:31:14 [single-1] INFO c.a.c.a.i.h.ConnectionHandler - onConnectionRemoteOpen hostname[some-namespace.servicebus.windows.net:5671], connectionId[MF_65d28d_1588604471389], remoteContainer[fa9aa3015eb84990a62817ad60423545_G8]
20:31:14 [single-1] INFO c.a.c.a.i.h.SessionHandler - onSessionRemoteOpen connectionId[MF_65d28d_1588604471389], entityName[dummy], sessionIncCapacity[0], sessionOutgoingWindow[2147483647]
20:31:14 [single-1] INFO c.a.c.a.i.ReactorConnection - Setting CBS channel.
20:31:14 [single-1] INFO c.a.c.a.i.ReactorConnection - Emitting new response channel. connectionId: MF_65d28d_1588604471389. entityPath: $cbs. linkName: cbs.
20:31:14 [single-1] INFO c.a.c.a.i.RequestResponseChannel<cbs-session> - connectionId[MF_65d28d_1588604471389] entityPath[$cbs]: Setting next AMQP channel.
20:31:14 [single-1] INFO c.a.c.a.i.h.ReceiveLinkHandler - onLinkLocalOpen connectionId[MF_65d28d_1588604471389], linkName[cbs:receiver], localSource[Source{address='$cbs', durable=NONE, expiryPolicy=SESSION_END, timeout=0, dynamic=false, dynamicNodeProperties=null, distributionMode=null, filter=null, defaultOutcome=null, outcomes=null, capabilities=null}]
20:31:14 [single-1] INFO c.a.c.a.i.h.SessionHandler - onSessionRemoteOpen connectionId[MF_65d28d_1588604471389], entityName[cbs-session], sessionIncCapacity[0], sessionOutgoingWindow[2147483647]
20:31:14 [single-1] INFO c.a.c.a.i.h.SendLinkHandler - onLinkRemoteOpen connectionId[MF_65d28d_1588604471389], linkName[cbs:sender], remoteTarget[Target{address='$cbs', durable=NONE, expiryPolicy=SESSION_END, timeout=0, dynamic=false, dynamicNodeProperties=null, capabilities=null}]
20:31:14 [single-1] INFO c.a.c.a.i.h.ReceiveLinkHandler - onLinkRemoteOpen connectionId[MF_65d28d_1588604471389], linkName[cbs:receiver], remoteSource[Source{address='$cbs', durable=NONE, expiryPolicy=SESSION_END, timeout=0, dynamic=false, dynamicNodeProperties=null, distributionMode=null, filter=null, defaultOutcome=null, outcomes=null, capabilities=null}]
20:31:14 [single-1] INFO c.a.c.a.i.ActiveClientTokenManager - Scheduling refresh token task. scopes[amqp://some-namespace.servicebus.windows.net/dummy]
20:31:14 [single-1] WARN c.a.c.a.i.RetryUtil - Error is not a TimeoutException nor is it a retryable AMQP exception.
Size of the payload exceeded maximum message size: 256 kb, errorContext[NAMESPACE: some-namespace.servicebus.windows.net, PATH: dummy, REFERENCE_ID: dummy, LINK_CREDIT: 0]
Size of the payload exceeded maximum message size: 256 kb, errorContext[NAMESPACE: some-namespace.servicebus.windows.net, PATH: dummy, REFERENCE_ID: dummy, LINK_CREDIT: 0]
Process finished with exit code 0
这可能会有所帮助。
我模拟了需要 30 多分钟(使用 31 分钟,只是为了安全)才能完全填满批次并发送的情况。 批次首次初始化的时间与批次完全填满的时间之间的差异必须 > 30 分钟。
请注意最后的日志,它说它不是 TimeoutException,而是说大小大于 256 KB,考虑到创建批处理时的最大批处理大小假定为 1022 KB,这不应该是这种情况。
@shubhambhattar我正在调查这个问题。 感谢您提供可重现的样本。 我很快就会有更新。