Aws-iot-device-sdk-python-v2: AWS_ERROR_MQTT_UNEXPECTED_HANGUP無限の再接続ループ
以下の[]を[x]に変更して、バグであることを確認してください。
- [x]以前の同様の問題を検索しましたが、解決策が見つかりませんでした
既知の問題
- [x] ATSデータ型エンドポイントを使用しています：エンドポイントは
<prefix>-ats.iot.<region>.amazonaws.comようになります
こんにちはAWSチーム、
2日以来SDKで直面している重大な問題に少し立ち往生しています（詳細は以下を参照）。
バグを説明する
ホスト解決直後のタスクスケジューラによるTLSチャネルのシャットダウンにより、無限のRECONNECTING <> AWS_ERROR_MQTT_UNEXPECTED_HANGUPループが発生します（大量のデータを食べる!!）。
エンドポイントへのSDK接続が確立されることはありません。 TLSのセッションの開始と終了が非常に速いペースで行われると、データ使用量が大幅に増加します（突然の問題により、フィールドデバイスで数百ドルのデータローミング料金が発生しました）。
SDKのバージョン番号
1.5.12と1.5.14でテストされ、それぞれの場合にaws-crtが一致します
プラットフォーム/ OS /デバイス
Armhfデバイス（Cortex-A7）
ubuntu20.04に基づくDockerコンテナ
再現する（観察された行動）
SDKの標準的な使用が観察され、再接続ループが発生する前にデバイスが10日間実行されていました。 その後、デバイスはエンドポイントに正しく再接続できなくなり、再接続ループが常に存在します。 通常の操作から10日後の2021年4月30日に動作が変更されました（接続と3分ごとの公開）。 証明書とCAの有効期限が切れておらず、新しい証明書でデバイスを更新しても役に立ちませんでした。 SDK1.5.12から1.15.14へのアップデートも役に立ちませんでした。
AWS_KEEP_ALIVE_SECS = 30
# setup AWS connection
io.init_logging(io.LogLevel.Debug, 'stderr')
event_loop_group = io.EventLoopGroup(1)
host_resolver = io.DefaultHostResolver(event_loop_group)
client_bootstrap = io.ClientBootstrap(event_loop_group, host_resolver)
client_id = "auto-%s" % str(uuid.uuid4()).upper()
aws_mqtt = mqtt_connection_builder.mtls_from_path(
endpoint=args.endpoint,
cert_filepath="%s/device.crt" % (args.provision_location),
pri_key_filepath="%s/device.key" % (args.provision_location),
client_bootstrap=client_bootstrap,
ca_filepath="%s/AmazonRootCA1.pem" % (args.cert_location),
on_connection_interrupted=aws_on_connection_interrupted,
on_connection_resumed=aws_on_connection_resumed,z
client_id=client_id,
clean_session=False,
keep_alive_secs=AWS_KEEP_ALIVE_SECS)
予想される行動
予期しないハングアップはありません。 そして、無限の再接続ループはありません。
ログ/出力
ネットワークの使用により、展開されたローミングデバイス（16GB）で数GBのSIMデータが使用されました。 eth0ベースの接続試行でのネットワークデータの使用量は次のとおりです。
デバッグレベルのトレース
[?2004l[?1l>[?25h[>4;m[?1049l[23;0;0t]0;root@aeler-imx7ulpea-ucom: /root@aeler-imx7ulpea-ucom:/# pyu thon3 - service.py
aws-iotsdk> [2021-05-04 13:53:48.925804] AWS Topic: /d/22f66ba6f37b3ede6115c86389e530eb1d408103
aws-iotsdk> [2021-05-04 13:53:48.952523] [mc] connecting to localhost on port 1883
aws-iotsdk> [2021-05-04 13:53:49.104120] [mc] connect rc: 0
[INFO] [2021-05-04T13:53:50Z] [b6f3f460] [event-loop] - id=0x1684fc8: Initializing edge-triggered epoll
[INFO] [2021-05-04T13:53:50Z] [b6f3f460] [event-loop] - id=0x1684fc8: Using eventfd for cross-thread notifications.
[INFO] [2021-05-04T13:53:50Z] [b6f3f460] [event-loop] - id=0x1684fc8: Starting event-loop thread.
[INFO] [2021-05-04T13:53:50Z] [b6f3f460] [dns] - id=0x162d7a0: Initializing default host resolver with 16 max host entries.
[INFO] [2021-05-04T13:53:50Z] [b6f3f460] [channel-bootstrap] - id=0x164f998: Initializing client bootstrap with event-loop group 0x16eddb0
[INFO] [2021-05-04T13:53:50Z] [b51fe460] [event-loop] - id=0x1684fc8: main loop started
[INFO] [2021-05-04T13:53:50Z] [b51fe460] [event-loop] - id=0x1684fc8: default timeout 100000, and max events to process per tick 100
[DEBUG] [2021-05-04T13:53:53Z] [b6f3f460] [tls-handler] - ctx: Certificate and key have been set, setting them up now.
[INFO] [2021-05-04T13:53:53Z] [b6f3f460] [tls-handler] - ctx: cannot enable ocsp stapling: OCSP stapling was requested, but is not supported
[DEBUG] [2021-05-04T13:53:53Z] [b6f3f460] [tls-handler] - ctx: Setting ALPN list x-amzn-mqtt-ca
[DEBUG] [2021-05-04T13:53:53Z] [b6f3f460] [mqtt-client] - client=0x17347e8: Initalizing MQTT client
[DEBUG] [2021-05-04T13:53:53Z] [b6f3f460] [mqtt-client] - id=0x163d9b8: Creating new connection
[DEBUG] [2021-05-04T13:53:53Z] [b6f3f460] [mqtt-topic-tree] - tree=0x163db24: Creating new topic tree
aws-iotsdk> [2021-05-04 13:53:53.308811] checking internet connection.
aws-iotsdk> [2021-05-04 13:53:53.734491] [aws] Connecting to <my_endpoint>-ats.iot.us-east-1.amazonaws.com as auto-2C288BE4-A555-4653-92D8-A632727D5B1C ...
[DEBUG] [2021-05-04T13:53:53Z] [b6f3f460] [mqtt-client] - id=0x163d9b8: Begin connecting process, switch state to CONNECTING.
[INFO] [2021-05-04T13:53:53Z] [b6f3f460] [mqtt-client] - id=0x163d9b8: using ping timeout of 3000000000 ns
[DEBUG] [2021-05-04T13:53:53Z] [b6f3f460] [dns] - id=0x162d7a0: Host resolution requested for <my_endpoint>-ats.iot.us-east-1.amazonaws.com
[DEBUG] [2021-05-04T13:53:53Z] [b6f3f460] [dns] - id=0x162d7a0: No cached entries found for <my_endpoint>-ats.iot.us-east-1.amazonaws.com starting new resolver thread.
[DEBUG] [2021-05-04T13:53:53Z] [b467f460] [dns] - static: resolving host <my_endpoint>-ats.iot.us-east-1.amazonaws.com
[DEBUG] [2021-05-04T13:53:53Z] [b467f460] [dns] - static: resolved record: 52.21.198.210
[DEBUG] [2021-05-04T13:53:53Z] [b467f460] [dns] - static: resolved record: 54.147.70.168
[DEBUG] [2021-05-04T13:53:53Z] [b467f460] [dns] - static: resolved record: 34.206.245.43
[DEBUG] [2021-05-04T13:53:53Z] [b467f460] [dns] - static: resolved record: 54.158.175.156
[DEBUG] [2021-05-04T13:53:53Z] [b467f460] [dns] - static: resolved record: 3.225.200.224
[DEBUG] [2021-05-04T13:53:53Z] [b467f460] [dns] - static: resolved record: 54.145.247.28
[DEBUG] [2021-05-04T13:53:53Z] [b467f460] [dns] - static: resolved record: 54.84.18.235
[DEBUG] [2021-05-04T13:53:53Z] [b467f460] [dns] - static: resolved record: 18.207.61.253
[DEBUG] [2021-05-04T13:53:53Z] [b467f460] [dns] - static: resolved record: 2406:da00:ff00::3e1:f981
[DEBUG] [2021-05-04T13:53:53Z] [b467f460] [dns] - static: resolved record: 2406:da00:ff00::3417:2507
[DEBUG] [2021-05-04T13:53:53Z] [b467f460] [dns] - static: resolved record: 2406:da00:ff00::23ad:4123
[DEBUG] [2021-05-04T13:53:53Z] [b467f460] [dns] - static: resolved record: 2406:da00:ff00::22cd:842b
[DEBUG] [2021-05-04T13:53:53Z] [b467f460] [dns] - static: resolved record: 2406:da00:ff00::3db:a390
[DEBUG] [2021-05-04T13:53:53Z] [b467f460] [dns] - static: resolved record: 2406:da00:ff00::34c8:5c29
[DEBUG] [2021-05-04T13:53:53Z] [b467f460] [dns] - static: resolved record: 2406:da00:ff00::36a3:c4c1
[DEBUG] [2021-05-04T13:53:53Z] [b467f460] [dns] - static: resolved record: 2406:da00:ff00::3e1:b950
[DEBUG] [2021-05-04T13:53:53Z] [b467f460] [dns] - static, resolving host <my_endpoint>-ats.iot.us-east-1.amazonaws.com successful, returned 16 addresses
[DEBUG] [2021-05-04T13:53:53Z] [b467f460] [dns] - static: new address resolved 52.21.198.210 for host <my_endpoint>-ats.iot.us-east-1.amazonaws.com caching
[DEBUG] [2021-05-04T13:53:53Z] [b467f460] [dns] - static: new address resolved 54.147.70.168 for host <my_endpoint>-ats.iot.us-east-1.amazonaws.com caching
[DEBUG] [2021-05-04T13:53:53Z] [b467f460] [dns] - static: new address resolved 34.206.245.43 for host <my_endpoint>-ats.iot.us-east-1.amazonaws.com caching
[DEBUG] [2021-05-04T13:53:53Z] [b467f460] [dns] - static: new address resolved 54.158.175.156 for host <my_endpoint>-ats.iot.us-east-1.amazonaws.com caching
[DEBUG] [2021-05-04T13:53:53Z] [b467f460] [dns] - static: new address resolved 3.225.200.224 for host <my_endpoint>-ats.iot.us-east-1.amazonaws.com caching
[DEBUG] [2021-05-04T13:53:53Z] [b467f460] [dns] - static: new address resolved 54.145.247.28 for host <my_endpoint>-ats.iot.us-east-1.amazonaws.com caching
[DEBUG] [2021-05-04T13:53:53Z] [b467f460] [dns] - static: new address resolved 54.84.18.235 for host <my_endpoint>-ats.iot.us-east-1.amazonaws.com caching
[DEBUG] [2021-05-04T13:53:53Z] [b467f460] [dns] - static: new address resolved 18.207.61.253 for host <my_endpoint>-ats.iot.us-east-1.amazonaws.com caching
[DEBUG] [2021-05-04T13:53:53Z] [b467f460] [dns] - static: new address resolved 2406:da00:ff00::3e1:f981 for host <my_endpoint>-ats.iot.us-east-1.amazonaws.com caching
[DEBUG] [2021-05-04T13:53:53Z] [b467f460] [dns] - static: new address resolved 2406:da00:ff00::3417:2507 for host <my_endpoint>-ats.iot.us-east-1.amazonaws.com caching
[DEBUG] [2021-05-04T13:53:53Z] [b467f460] [dns] - static: new address resolved 2406:da00:ff00::23ad:4123 for host <my_endpoint>-ats.iot.us-east-1.amazonaws.com caching
[DEBUG] [2021-05-04T13:53:53Z] [b467f460] [dns] - static: new address resolved 2406:da00:ff00::22cd:842b for host <my_endpoint>-ats.iot.us-east-1.amazonaws.com caching
[DEBUG] [2021-05-04T13:53:53Z] [b467f460] [dns] - static: new address resolved 2406:da00:ff00::3db:a390 for host <my_endpoint>-ats.iot.us-east-1.amazonaws.com caching
[DEBUG] [2021-05-04T13:53:53Z] [b467f460] [dns] - static: new address resolved 2406:da00:ff00::34c8:5c29 for host <my_endpoint>-ats.iot.us-east-1.amazonaws.com caching
[DEBUG] [2021-05-04T13:53:53Z] [b467f460] [dns] - static: new address resolved 2406:da00:ff00::36a3:c4c1 for host <my_endpoint>-ats.iot.us-east-1.amazonaws.com caching
[DEBUG] [2021-05-04T13:53:53Z] [b467f460] [dns] - static: new address resolved 2406:da00:ff00::3e1:b950 for host <my_endpoint>-ats.iot.us-east-1.amazonaws.com caching
[DEBUG] [2021-05-04T13:53:53Z] [b467f460] [dns] - static, invoking resolution callback for host <my_endpoint>-ats.iot.us-east-1.amazonaws.com with 2 addresses
[DEBUG] [2021-05-04T13:53:53Z] [b51fe460] [task-scheduler] - id=0xb4804c88: Scheduling attempt_connection task for immediate execution
[DEBUG] [2021-05-04T13:53:53Z] [b51fe460] [task-scheduler] - id=0xb4804c88: Running attempt_connection task with <Running> status
[DEBUG] [2021-05-04T13:53:53Z] [b51fe460] [socket] - id=0xb4805118 fd=10: initializing with domain 1 and type 0
[DEBUG] [2021-05-04T13:53:53Z] [b51fe460] [socket] - id=0xb4805118 fd=10: setting socket options to: keep-alive 0, keep idle 0, keep-alive interval 0, keep-alive probe count 0.
[WARN] [2021-05-04T13:53:53Z] [b51fe460] [socket] - id=0xb4805118 fd=10: setsockopt() for NO_SIGNAL failed with errno 92. If you are having SIGPIPE signals thrown, you may want to install a signal trap in your application layer.
[DEBUG] [2021-05-04T13:53:53Z] [b51fe460] [socket] - id=0xb4805118 fd=10: beginning connect.
[DEBUG] [2021-05-04T13:53:53Z] [b51fe460] [socket] - id=0xb4805118 fd=10: connecting to endpoint 2406:da00:ff00::3e1:f981:443.
[ERROR] [2021-05-04T13:53:53Z] [b51fe460] [socket] - id=0xb4805118 fd=10: connect failed with error code 101.
[INFO] [2021-05-04T13:53:53Z] [b51fe460] [dns] - id=0x162d7a0: recording failure for record 2406:da00:ff00::3e1:f981 for <my_endpoint>-ats.iot.us-east-1.amazonaws.com, moving to bad list
[DEBUG] [2021-05-04T13:53:53Z] [b51fe460] [dns] - static: purging address 2406:da00:ff00::3e1:f981 for host <my_endpoint>-ats.iot.us-east-1.amazonaws.com from the cache due to cache eviction or shutdown
[DEBUG] [2021-05-04T13:53:53Z] [b51fe460] [socket] - id=0xb4805118 fd=10: is still open, closing...
[DEBUG] [2021-05-04T13:53:53Z] [b51fe460] [socket] - id=0xb4805118 fd=10: closing
[ERROR] [2021-05-04T13:53:53Z] [b51fe460] [channel-bootstrap] - id=0x164f998: failed to create socket with error 1049
[DEBUG] [2021-05-04T13:53:53Z] [b51fe460] [task-scheduler] - id=0xb4804de0: Scheduling attempt_connection task for immediate execution
[DEBUG] [2021-05-04T13:53:53Z] [b51fe460] [task-scheduler] - id=0xb4804de0: Running attempt_connection task with <Running> status
[DEBUG] [2021-05-04T13:53:53Z] [b51fe460] [socket] - id=0xb4805118 fd=10: initializing with domain 0 and type 0
[DEBUG] [2021-05-04T13:53:53Z] [b51fe460] [socket] - id=0xb4805118 fd=10: setting socket options to: keep-alive 0, keep idle 0, keep-alive interval 0, keep-alive probe count 0.
[WARN] [2021-05-04T13:53:53Z] [b51fe460] [socket] - id=0xb4805118 fd=10: setsockopt() for NO_SIGNAL failed with errno 92. If you are having SIGPIPE signals thrown, you may want to install a signal trap in your application layer.
[DEBUG] [2021-05-04T13:53:53Z] [b51fe460] [socket] - id=0xb4805118 fd=10: beginning connect.
[DEBUG] [2021-05-04T13:53:53Z] [b51fe460] [socket] - id=0xb4805118 fd=10: connecting to endpoint 52.21.198.210:443.
[DEBUG] [2021-05-04T13:53:53Z] [b51fe460] [task-scheduler] - id=0xb4803a48: Scheduling (null) task for future execution at time 2076022788913
[DEBUG] [2021-05-04T13:53:54Z] [b51fe460] [task-scheduler] - id=0xb4805628: Scheduling epoll_event_loop_unsubscribe_cleanup task for immediate execution
[INFO] [2021-05-04T13:53:54Z] [b51fe460] [socket] - id=0xb4805118 fd=10: connection success
[DEBUG] [2021-05-04T13:53:54Z] [b51fe460] [socket] - id=0xb4805118 fd=10: local endpoint 10.42.0.177:39232
[DEBUG] [2021-05-04T13:53:54Z] [b51fe460] [socket] - id=0xb4805118 fd=10: assigning to event loop 0x1684fc8
[DEBUG] [2021-05-04T13:53:54Z] [b51fe460] [channel-bootstrap] - id=0x164f998: client connection on socket 0xb4805118 completed with error 0.
[DEBUG] [2021-05-04T13:53:54Z] [b51fe460] [channel] - id=0xb4806838: Beginning creation and setup of new channel.
[DEBUG] [2021-05-04T13:53:54Z] [b51fe460] [task-scheduler] - id=0xb4804ef8: Scheduling on_channel_setup_complete task for immediate execution
[DEBUG] [2021-05-04T13:53:54Z] [b51fe460] [task-scheduler] - id=0xb4805628: Running epoll_event_loop_unsubscribe_cleanup task with <Running> status
[DEBUG] [2021-05-04T13:53:54Z] [b51fe460] [task-scheduler] - id=0xb4804ef8: Running on_channel_setup_complete task with <Running> status
[DEBUG] [2021-05-04T13:53:54Z] [b51fe460] [channel] - id=0xb4806838: setup complete, notifying caller.
[DEBUG] [2021-05-04T13:53:54Z] [b51fe460] [channel] - id=0xb4806838: no message pool is currently stored in the event-loop local storage, adding 0xb4805c40 with max message size 16384, message count 4, with 4 small blocks of 128 bytes.
[DEBUG] [2021-05-04T13:53:54Z] [b51fe460] [channel-bootstrap] - id=0x164f998: channel 0xb4806838 setup succeeded: bootstrapping.
[DEBUG] [2021-05-04T13:53:54Z] [b51fe460] [socket-handler] - id=0xb4817418: Socket handler created with max_read_size of 16384
[DEBUG] [2021-05-04T13:53:54Z] [b51fe460] [task-scheduler] - id=0xb48175e8: Scheduling tls_timeout task for future execution at time 2081199692246
[DEBUG] [2021-05-04T13:53:54Z] [b51fe460] [task-scheduler] - id=0xb4805260: Scheduling socket_written_task task for immediate execution
[DEBUG] [2021-05-04T13:53:54Z] [b51fe460] [task-scheduler] - id=0xb4805260: Running socket_written_task task with <Running> status
[DEBUG] [2021-05-04T13:53:54Z] [b51fe460] [task-scheduler] - id=0xb4805260: Scheduling socket_written_task task for immediate execution
[DEBUG] [2021-05-04T13:53:54Z] [b51fe460] [task-scheduler] - id=0xb4805260: Running socket_written_task task with <Running> status
[DEBUG] [2021-05-04T13:53:54Z] [b51fe460] [tls-handler] - id=0xb48175d0: Alpn protocol negotiated as x-amzn-mqtt-ca
[DEBUG] [2021-05-04T13:53:54Z] [b51fe460] [tls-handler] - id=0xb48175d0: Remote server name is <my_endpoint>-ats.iot.us-east-1.amazonaws.com
[DEBUG] [2021-05-04T13:53:54Z] [b51fe460] [channel-bootstrap] - id=0x164f998: tls negotiation result 0 on channel 0xb4806838
[DEBUG] [2021-05-04T13:53:54Z] [b51fe460] [mqtt-client] - id=0x163d9b8: Connection successfully opened, sending CONNECT packet
[DEBUG] [2021-05-04T13:53:54Z] [b51fe460] [task-scheduler] - id=0xb48349a0: Scheduling mqtt_connack_timeout task for future execution at time 2074947672913
[DEBUG] [2021-05-04T13:53:54Z] [b51fe460] [mqtt-client] - id=0x163d9b8: Adding username ?SDK=PythonV2&Version=dev to connection
[DEBUG] [2021-05-04T13:53:54Z] [b51fe460] [task-scheduler] - id=0xb4805260: Scheduling socket_written_task task for immediate execution
[DEBUG] [2021-05-04T13:53:54Z] [b51fe460] [task-scheduler] - id=0xb4805260: Running socket_written_task task with <Running> status
[DEBUG] [2021-05-04T13:53:54Z] [b467f460] [dns] - static: resolving host <my_endpoint>-ats.iot.us-east-1.amazonaws.com
[DEBUG] [2021-05-04T13:53:54Z] [b467f460] [dns] - static: resolved record: 18.207.61.253
[DEBUG] [2021-05-04T13:53:54Z] [b467f460] [dns] - static: resolved record: 52.21.198.210
[DEBUG] [2021-05-04T13:53:54Z] [b467f460] [dns] - static: resolved record: 54.147.70.168
[DEBUG] [2021-05-04T13:53:54Z] [b467f460] [dns] - static: resolved record: 34.206.245.43
[DEBUG] [2021-05-04T13:53:54Z] [b467f460] [dns] - static: resolved record: 54.158.175.156
[DEBUG] [2021-05-04T13:53:54Z] [b467f460] [dns] - static: resolved record: 3.225.200.224
[DEBUG] [2021-05-04T13:53:54Z] [b467f460] [dns] - static: resolved record: 54.145.247.28
[DEBUG] [2021-05-04T13:53:54Z] [b467f460] [dns] - static: resolved record: 54.84.18.235
[DEBUG] [2021-05-04T13:53:54Z] [b467f460] [dns] - static: resolved record: 2406:da00:ff00::3e1:b950
[DEBUG] [2021-05-04T13:53:54Z] [b467f460] [dns] - static: resolved record: 2406:da00:ff00::3e1:f981
[DEBUG] [2021-05-04T13:53:54Z] [b467f460] [dns] - static: resolved record: 2406:da00:ff00::3417:2507
[DEBUG] [2021-05-04T13:53:54Z] [b467f460] [dns] - static: resolved record: 2406:da00:ff00::23ad:4123
[DEBUG] [2021-05-04T13:53:54Z] [b467f460] [dns] - static: resolved record: 2406:da00:ff00::22cd:842b
[DEBUG] [2021-05-04T13:53:54Z] [b467f460] [dns] - static: resolved record: 2406:da00:ff00::3db:a390
[DEBUG] [2021-05-04T13:53:54Z] [b467f460] [dns] - static: resolved record: 2406:da00:ff00::34c8:5c29
[DEBUG] [2021-05-04T13:53:54Z] [b467f460] [dns] - static: resolved record: 2406:da00:ff00::36a3:c4c1
[DEBUG] [2021-05-04T13:53:54Z] [b467f460] [dns] - static, resolving host <my_endpoint>-ats.iot.us-east-1.amazonaws.com successful, returned 16 addresses
[DEBUG] [2021-05-04T13:53:55Z] [b51fe460] [mqtt-client] - id=0x163d9b8: connection was accepted, switch state from 0 to CONNECTED.
[DEBUG] [2021-05-04T13:53:55Z] [b51fe460] [task-scheduler] - id=0x163dac8: Scheduling mqtt_ping task for future execution at time 2102147419913
[DEBUG] [2021-05-04T13:53:55Z] [b467f460] [dns] - static: resolving host <my_endpoint>-ats.iot.us-east-1.amazonaws.com
[DEBUG] [2021-05-04T13:53:55Z] [b467f460] [dns] - static: resolved record: 54.84.18.235
[DEBUG] [2021-05-04T13:53:55Z] [b467f460] [dns] - static: resolved record: 18.207.61.253
[DEBUG] [2021-05-04T13:53:55Z] [b467f460] [dns] - static: resolved record: 52.21.198.210
[DEBUG] [2021-05-04T13:53:55Z] [b467f460] [dns] - static: resolved record: 54.147.70.168
[DEBUG] [2021-05-04T13:53:55Z] [b467f460] [dns] - static: resolved record: 34.206.245.43
[DEBUG] [2021-05-04T13:53:55Z] [b467f460] [dns] - static: resolved record: 54.158.175.156
[DEBUG] [2021-05-04T13:53:55Z] [b467f460] [dns] - static: resolved record: 3.225.200.224
[DEBUG] [2021-05-04T13:53:55Z] [b467f460] [dns] - static: resolved record: 54.145.247.28
[DEBUG] [2021-05-04T13:53:55Z] [b467f460] [dns] - static: resolved record: 2406:da00:ff00::36a3:c4c1
[DEBUG] [2021-05-04T13:53:55Z] [b467f460] [dns] - static: resolved record: 2406:da00:ff00::3e1:b950
[DEBUG] [2021-05-04T13:53:55Z] [b467f460] [dns] - static: resolved record: 2406:da00:ff00::3e1:f981
[DEBUG] [2021-05-04T13:53:55Z] [b467f460] [dns] - static: resolved record: 2406:da00:ff00::3417:2507
[DEBUG] [2021-05-04T13:53:55Z] [b467f460] [dns] - static: resolved record: 2406:da00:ff00::23ad:4123
[DEBUG] [2021-05-04T13:53:55Z] [b467f460] [dns] - static: resolved record: 2406:da00:ff00::22cd:842b
[DEBUG] [2021-05-04T13:53:55Z] [b467f460] [dns] - static: resolved record: 2406:da00:ff00::3db:a390
[DEBUG] [2021-05-04T13:53:55Z] [b467f460] [dns] - static: resolved record: 2406:da00:ff00::34c8:5c29
[DEBUG] [2021-05-04T13:53:55Z] [b467f460] [dns] - static, resolving host <my_endpoint>-ats.iot.us-east-1.amazonaws.com successful, returned 16 addresses
aws-iotsdk> [2021-05-04 13:53:56.689408] checking internet connection.
[DEBUG] [2021-05-04T13:53:56Z] [b467f460] [dns] - static: resolving host <my_endpoint>-ats.iot.us-east-1.amazonaws.com
[DEBUG] [2021-05-04T13:53:56Z] [b467f460] [dns] - static: resolved record: 3.225.200.224
[DEBUG] [2021-05-04T13:53:56Z] [b467f460] [dns] - static: resolved record: 54.145.247.28
[DEBUG] [2021-05-04T13:53:56Z] [b467f460] [dns] - static: resolved record: 54.84.18.235
[DEBUG] [2021-05-04T13:53:56Z] [b467f460] [dns] - static: resolved record: 18.207.61.253
[DEBUG] [2021-05-04T13:53:56Z] [b467f460] [dns] - static: resolved record: 52.21.198.210
[DEBUG] [2021-05-04T13:53:56Z] [b467f460] [dns] - static: resolved record: 54.147.70.168
[DEBUG] [2021-05-04T13:53:56Z] [b467f460] [dns] - static: resolved record: 34.206.245.43
[DEBUG] [2021-05-04T13:53:57Z] [b467f460] [dns] - static: resolved record: 54.158.175.156
[DEBUG] [2021-05-04T13:53:57Z] [b467f460] [dns] - static: resolved record: 2406:da00:ff00::3db:a390
[DEBUG] [2021-05-04T13:53:57Z] [b467f460] [dns] - static: resolved record: 2406:da00:ff00::34c8:5c29
[DEBUG] [2021-05-04T13:53:57Z] [b467f460] [dns] - static: resolved record: 2406:da00:ff00::36a3:c4c1
[DEBUG] [2021-05-04T13:53:57Z] [b467f460] [dns] - static: resolved record: 2406:da00:ff00::3e1:b950
[DEBUG] [2021-05-04T13:53:57Z] [b467f460] [dns] - static: resolved record: 2406:da00:ff00::3e1:f981
[DEBUG] [2021-05-04T13:53:57Z] [b467f460] [dns] - static: resolved record: 2406:da00:ff00::3417:2507
[DEBUG] [2021-05-04T13:53:57Z] [b467f460] [dns] - static: resolved record: 2406:da00:ff00::23ad:4123
[DEBUG] [2021-05-04T13:53:57Z] [b467f460] [dns] - static: resolved record: 2406:da00:ff00::22cd:842b
[DEBUG] [2021-05-04T13:53:57Z] [b467f460] [dns] - static, resolving host <my_endpoint>-ats.iot.us-east-1.amazonaws.com successful, returned 16 addresses
aws-iotsdk> [2021-05-04 13:53:57.606042] [aws] Publishing message to topic '/d/22f66ba6f37b3ede6115c86389e530eb1d408103
'
[DEBUG] [2021-05-04T13:53:57Z] [b51fe460] [task-scheduler] - id=0xb4806950: Scheduling schedule_cross_thread_tasks task for immediate execution
[DEBUG] [2021-05-04T13:53:57Z] [b51fe460] [task-scheduler] - id=0xb4806950: Running schedule_cross_thread_tasks task with <Running> status
[DEBUG] [2021-05-04T13:53:57Z] [b51fe460] [task-scheduler] - id=0xb4805260: Scheduling socket_written_task task for immediate execution
[DEBUG] [2021-05-04T13:53:57Z] [b6314460] [mqtt-client] - id=0x163d9b8: Starting publish 1 to topic /d/22f66ba6f37b3ede6115c86389e530eb1d408103
[DEBUG] [2021-05-04T13:53:57Z] [b51fe460] [task-scheduler] - id=0xb4805260: Running socket_written_task task with <Running> status
[DEBUG] [2021-05-04T13:53:57Z] [b51fe460] [task-scheduler] - id=0xb48349a0: Running mqtt_connack_timeout task with <Running> status
[DEBUG] [2021-05-04T13:53:57Z] [b51fe460] [task-scheduler] - id=0xb4805260: Scheduling socket_written_task task for immediate execution
[DEBUG] [2021-05-04T13:53:57Z] [b51fe460] [task-scheduler] - id=0xb4805260: Running socket_written_task task with <Running> status
[DEBUG] [2021-05-04T13:53:57Z] [b51fe460] [task-scheduler] - id=0xb4805260: Scheduling socket_written_task task for immediate execution
[DEBUG] [2021-05-04T13:53:57Z] [b51fe460] [task-scheduler] - id=0xb4805260: Running socket_written_task task with <Running> status
[DEBUG] [2021-05-04T13:53:58Z] [b467f460] [dns] - static: resolving host <my_endpoint>-ats.iot.us-east-1.amazonaws.com
[DEBUG] [2021-05-04T13:53:58Z] [b467f460] [dns] - static: resolved record: 54.158.175.156
[DEBUG] [2021-05-04T13:53:58Z] [b467f460] [dns] - static: resolved record: 3.225.200.224
[DEBUG] [2021-05-04T13:53:58Z] [b467f460] [dns] - static: resolved record: 54.145.247.28
[DEBUG] [2021-05-04T13:53:58Z] [b467f460] [dns] - static: resolved record: 54.84.18.235
[DEBUG] [2021-05-04T13:53:58Z] [b467f460] [dns] - static: resolved record: 18.207.61.253
[DEBUG] [2021-05-04T13:53:58Z] [b467f460] [dns] - static: resolved record: 52.21.198.210
[DEBUG] [2021-05-04T13:53:58Z] [b467f460] [dns] - static: resolved record: 54.147.70.168
[DEBUG] [2021-05-04T13:53:58Z] [b467f460] [dns] - static: resolved record: 34.206.245.43
[DEBUG] [2021-05-04T13:53:58Z] [b467f460] [dns] - static: resolved record: 2406:da00:ff00::22cd:842b
[DEBUG] [2021-05-04T13:53:58Z] [b467f460] [dns] - static: resolved record: 2406:da00:ff00::3db:a390
[DEBUG] [2021-05-04T13:53:58Z] [b467f460] [dns] - static: resolved record: 2406:da00:ff00::34c8:5c29
[DEBUG] [2021-05-04T13:53:58Z] [b467f460] [dns] - static: resolved record: 2406:da00:ff00::36a3:c4c1
[DEBUG] [2021-05-04T13:53:58Z] [b467f460] [dns] - static: resolved record: 2406:da00:ff00::3e1:b950
[DEBUG] [2021-05-04T13:53:58Z] [b467f460] [dns] - static: resolved record: 2406:da00:ff00::3e1:f981
[DEBUG] [2021-05-04T13:53:58Z] [b467f460] [dns] - static: resolved record: 2406:da00:ff00::3417:2507
[DEBUG] [2021-05-04T13:53:58Z] [b467f460] [dns] - static: resolved record: 2406:da00:ff00::23ad:4123
[DEBUG] [2021-05-04T13:53:58Z] [b467f460] [dns] - static, resolving host <my_endpoint>-ats.iot.us-east-1.amazonaws.com successful, returned 16 addresses
[DEBUG] [2021-05-04T13:53:58Z] [b51fe460] [task-scheduler] - id=0xb4805260: Scheduling socket_written_task task for immediate execution
[DEBUG] [2021-05-04T13:53:58Z] [b51fe460] [task-scheduler] - id=0xb4805260: Running socket_written_task task with <Running> status
[DEBUG] [2021-05-04T13:53:58Z] [b51fe460] [tls-handler] - id=0xb48175d0: Alert code 0
[DEBUG] [2021-05-04T13:53:58Z] [b51fe460] [task-scheduler] - id=0xb4806978: Scheduling channel_shutdown task for immediate execution
[DEBUG] [2021-05-04T13:53:58Z] [b51fe460] [task-scheduler] - id=0xb4806978: Running channel_shutdown task with <Running> status
[DEBUG] [2021-05-04T13:53:58Z] [b51fe460] [channel] - id=0xb4806838: beginning shutdown process
[DEBUG] [2021-05-04T13:53:58Z] [b51fe460] [channel] - id=0xb4806838: handler 0xb4817418 shutdown in read dir completed.
[DEBUG] [2021-05-04T13:53:58Z] [b51fe460] [tls-handler] - id=0xb48175d0: Shutting down read direction with error code 0
[DEBUG] [2021-05-04T13:53:58Z] [b51fe460] [channel] - id=0xb4806838: handler 0xb48175d0 shutdown in read dir completed.
[DEBUG] [2021-05-04T13:53:58Z] [b51fe460] [channel] - id=0xb4806838: handler 0x163d9cc shutdown in read dir completed.
[DEBUG] [2021-05-04T13:53:58Z] [b51fe460] [task-scheduler] - id=0xb4806850: Scheduling (null) task for immediate execution
[DEBUG] [2021-05-04T13:53:58Z] [b51fe460] [task-scheduler] - id=0xb4806850: Running (null) task with <Running> status
[INFO] [2021-05-04T13:53:58Z] [b51fe460] [mqtt-client] - id=0x163d9b8: sending disconnect message as part of graceful shutdown.
[DEBUG] [2021-05-04T13:53:58Z] [b51fe460] [mqtt-client] - id=0x163d9b8: failed to send courteous disconnect io message
[DEBUG] [2021-05-04T13:53:58Z] [b51fe460] [channel] - id=0xb4806838: handler 0x163d9cc shutdown in write dir completed.
[DEBUG] [2021-05-04T13:53:58Z] [b51fe460] [tls-handler] - id=0xb48175d0: Scheduling delayed write direction shutdown
[DEBUG] [2021-05-04T13:53:58Z] [b51fe460] [task-scheduler] - id=0xb48176e0: Scheduling s2n_delayed_shutdown task for future execution at time 2075467825247
[DEBUG] [2021-05-04T13:53:58Z] [b51fe460] [task-scheduler] - id=0xb48176e0: Running s2n_delayed_shutdown task with <Running> status
[DEBUG] [2021-05-04T13:53:58Z] [b51fe460] [tls-handler] - id=0xb48175d0: Delayed shut down in write direction
[DEBUG] [2021-05-04T13:53:58Z] [b51fe460] [task-scheduler] - id=0xb4805260: Scheduling socket_written_task task for immediate execution
[DEBUG] [2021-05-04T13:53:58Z] [b51fe460] [channel] - id=0xb4806838: handler 0xb48175d0 shutdown in write dir completed.
[DEBUG] [2021-05-04T13:53:58Z] [b51fe460] [socket] - id=0xb4805118 fd=10: closing
[DEBUG] [2021-05-04T13:53:58Z] [b51fe460] [task-scheduler] - id=0xb4805680: Scheduling epoll_event_loop_unsubscribe_cleanup task for immediate execution
[DEBUG] [2021-05-04T13:53:58Z] [b51fe460] [task-scheduler] - id=0xb4805260: Running socket_written_task task with <Canceled> status
[DEBUG] [2021-05-04T13:53:58Z] [b51fe460] [task-scheduler] - id=0xb4817478: Scheduling socket_handler_close task for immediate execution
[DEBUG] [2021-05-04T13:53:58Z] [b51fe460] [task-scheduler] - id=0xb4805680: Running epoll_event_loop_unsubscribe_cleanup task with <Running> status
[DEBUG] [2021-05-04T13:53:58Z] [b51fe460] [task-scheduler] - id=0xb4817478: Running socket_handler_close task with <Running> status
[DEBUG] [2021-05-04T13:53:58Z] [b51fe460] [channel] - id=0xb4806838: handler 0xb4817418 shutdown in write dir completed.
[DEBUG] [2021-05-04T13:53:58Z] [b51fe460] [task-scheduler] - id=0xb4806850: Scheduling (null) task for immediate execution
[DEBUG] [2021-05-04T13:53:58Z] [b51fe460] [task-scheduler] - id=0xb4806850: Running (null) task with <Running> status
[DEBUG] [2021-05-04T13:53:58Z] [b51fe460] [channel] - id=0xb4806838: during shutdown, canceling task 0xb48175e8
[DEBUG] [2021-05-04T13:53:58Z] [b51fe460] [task-scheduler] - id=0xb48175e8: Running tls_timeout task with <Canceled> status
[DEBUG] [2021-05-04T13:53:58Z] [b51fe460] [channel] - id=0xb4806838: during shutdown, canceling task 0x163dac8
[DEBUG] [2021-05-04T13:53:58Z] [b51fe460] [task-scheduler] - id=0x163dac8: Running mqtt_ping task with <Canceled> status
[DEBUG] [2021-05-04T13:53:58Z] [b51fe460] [channel-bootstrap] - id=0x164f998: channel 0xb4806838 shutdown with error 0.
[DEBUG] [2021-05-04T13:53:58Z] [b51fe460] [mqtt-client] - id=0x163d9b8: connection was unexpected interrupted, switch state to RECONNECTING.
[DEBUG] [2021-05-04T13:53:58Z] [b51fe460] [mqtt-client] - id=0x163d9b8: Connection interrupted, calling callback and attempting reconnect
aws-iotsdk> [2021-05-04 13:53:58.359515] [aws] Connection interrupted. error: AWS_ERROR_MQTT_UNEXPECTED_HANGUP: The connection was closed unexpectedly.
[DEBUG] [2021-05-04T13:53:58Z] [b51fe460] [dns] - id=0x162d7a0: Host resolution requested for <my_endpoint>-ats.iot.us-east-1.amazonaws.com
[DEBUG] [2021-05-04T13:53:58Z] [b51fe460] [dns] - id=0x162d7a0: cached entries found for <my_endpoint>-ats.iot.us-east-1.amazonaws.com returning to caller.
[DEBUG] [2021-05-04T13:53:58Z] [b51fe460] [task-scheduler] - id=0xb4834048: Scheduling attempt_connection task for immediate execution
[DEBUG] [2021-05-04T13:53:58Z] [b51fe460] [task-scheduler] - id=0xb48338c0: Scheduling attempt_connection task for immediate execution
[DEBUG] [2021-05-04T13:53:58Z] [b51fe460] [channel] - id=0xb4806838: destroying channel.
[DEBUG] [2021-05-04T13:53:58Z] [b51fe460] [channel-bootstrap] - id=0x164f998: releasing bootstrap reference
[DEBUG] [2021-05-04T13:53:58Z] [b51fe460] [task-scheduler] - id=0xb4834048: Running attempt_connection task with <Running> status
[DEBUG] [2021-05-04T13:53:58Z] [b51fe460] [socket] - id=0xb4805118 fd=10: initializing with domain 1 and type 0
[DEBUG] [2021-05-04T13:53:58Z] [b51fe460] [socket] - id=0xb4805118 fd=10: setting socket options to: keep-alive 0, keep idle 0, keep-alive interval 0, keep-alive probe count 0.
[WARN] [2021-05-04T13:53:58Z] [b51fe460] [socket] - id=0xb4805118 fd=10: setsockopt() for NO_SIGNAL failed with errno 92. If you are having SIGPIPE signals thrown, you may want to install a signal trap in your application layer.
[DEBUG] [2021-05-04T13:53:58Z] [b51fe460] [socket] - id=0xb4805118 fd=10: beginning connect.
[DEBUG] [2021-05-04T13:53:58Z] [b51fe460] [socket] - id=0xb4805118 fd=10: connecting to endpoint 2406:da00:ff00::22cd:842b:443.
[ERROR] [2021-05-04T13:53:58Z] [b51fe460] [socket] - id=0xb4805118 fd=10: connect failed with error code 101.
[INFO] [2021-05-04T13:53:58Z] [b51fe460] [dns] - id=0x162d7a0: recording failure for record 2406:da00:ff00::22cd:842b for <my_endpoint>-ats.iot.us-east-1.amazonaws.com, moving to bad list
[DEBUG] [2021-05-04T13:53:58Z] [b51fe460] [dns] - static: purging address 2406:da00:ff00::22cd:842b for host <my_endpoint>-ats.iot.us-east-1.amazonaws.com from the cache due to cache eviction or shutdown
[DEBUG] [2021-05-04T13:53:58Z] [b51fe460] [socket] - id=0xb4805118 fd=10: is still open, closing...
[DEBUG] [2021-05-04T13:53:58Z] [b51fe460] [socket] - id=0xb4805118 fd=10: closing
[ERROR] [2021-05-04T13:53:58Z] [b51fe460] [channel-bootstrap] - id=0x164f998: failed to create socket with error 1049
[DEBUG] [2021-05-04T13:53:58Z] [b51fe460] [task-scheduler] - id=0xb48338c0: Running attempt_connection task with <Running> status
[DEBUG] [2021-05-04T13:53:58Z] [b51fe460] [socket] - id=0xb4805118 fd=10: initializing with domain 0 and type 0
[DEBUG] [2021-05-04T13:53:58Z] [b51fe460] [socket] - id=0xb4805118 fd=10: setting socket options to: keep-alive 0, keep idle 0, keep-alive interval 0, keep-alive probe count 0.
[WARN] [2021-05-04T13:53:58Z] [b51fe460] [socket] - id=0xb4805118 fd=10: setsockopt() for NO_SIGNAL failed with errno 92. If you are having SIGPIPE signals thrown, you may want to install a signal trap in your application layer.
[DEBUG] [2021-05-04T13:53:58Z] [b51fe460] [socket] - id=0xb4805118 fd=10: beginning connect.
[DEBUG] [2021-05-04T13:53:58Z] [b51fe460] [socket] - id=0xb4805118 fd=10: connecting to endpoint 54.158.175.156:443.
[DEBUG] [2021-05-04T13:53:58Z] [b51fe460] [task-scheduler] - id=0xb4804268: Scheduling (null) task for future execution at time 2080568943914
[DEBUG] [2021-05-04T13:53:58Z] [b51fe460] [task-scheduler] - id=0xb482bd18: Scheduling epoll_event_loop_unsubscribe_cleanup task for immediate execution
[INFO] [2021-05-04T13:53:58Z] [b51fe460] [socket] - id=0xb4805118 fd=10: connection success
[DEBUG] [2021-05-04T13:53:58Z] [b51fe460] [socket] - id=0xb4805118 fd=10: local endpoint 10.42.0.177:47820
[DEBUG] [2021-05-04T13:53:58Z] [b51fe460] [socket] - id=0xb4805118 fd=10: assigning to event loop 0x1684fc8
[DEBUG] [2021-05-04T13:53:58Z] [b51fe460] [channel-bootstrap] - id=0x164f998: client connection on socket 0xb4805118 completed with error 0.
[DEBUG] [2021-05-04T13:53:58Z] [b51fe460] [channel] - id=0xb4806838: Beginning creation and setup of new channel.
[DEBUG] [2021-05-04T13:53:58Z] [b51fe460] [task-scheduler] - id=0xb4801cc8: Scheduling on_channel_setup_complete task for immediate execution
[DEBUG] [2021-05-04T13:53:58Z] [b51fe460] [task-scheduler] - id=0xb482bd18: Running epoll_event_loop_unsubscribe_cleanup task with <Running> status
[DEBUG] [2021-05-04T13:53:58Z] [b51fe460] [task-scheduler] - id=0xb4801cc8: Running on_channel_setup_complete task with <Running> status
[DEBUG] [2021-05-04T13:53:58Z] [b51fe460] [channel] - id=0xb4806838: setup complete, notifying caller.
[DEBUG] [2021-05-04T13:53:58Z] [b51fe460] [channel] - id=0xb4806838: message pool 0xb4805c40 found in event-loop local storage: using it.
[DEBUG] [2021-05-04T13:53:58Z] [b51fe460] [channel-bootstrap] - id=0x164f998: channel 0xb4806838 setup succeeded: bootstrapping.
[DEBUG] [2021-05-04T13:53:58Z] [b51fe460] [socket-handler] - id=0xb482ec00: Socket handler created with max_read_size of 16384
[DEBUG] [2021-05-04T13:53:58Z] [b51fe460] [task-scheduler] - id=0xb48175e8: Scheduling tls_timeout task for future execution at time 2085748031247
[DEBUG] [2021-05-04T13:53:58Z] [b51fe460] [task-scheduler] - id=0xb4805260: Scheduling socket_written_task task for immediate execution
[DEBUG] [2021-05-04T13:53:58Z] [b51fe460] [task-scheduler] - id=0xb4805260: Running socket_written_task task with <Running> status
[DEBUG] [2021-05-04T13:53:58Z] [b51fe460] [task-scheduler] - id=0xb4805260: Scheduling socket_written_task task for immediate execution
[DEBUG] [2021-05-04T13:53:58Z] [b51fe460] [task-scheduler] - id=0xb4805260: Running socket_written_task task with <Running> status
[DEBUG] [2021-05-04T13:53:59Z] [b51fe460] [task-scheduler] - id=0xb4803a48: Running (null) task with <Running> status
[DEBUG] [2021-05-04T13:53:59Z] [b467f460] [dns] - static: resolving host <my_endpoint>-ats.iot.us-east-1.amazonaws.com
[DEBUG] [2021-05-04T13:53:59Z] [b467f460] [dns] - static: resolved record: 34.206.245.43
[DEBUG] [2021-05-04T13:53:59Z] [b467f460] [dns] - static: resolved record: 54.158.175.156
[DEBUG] [2021-05-04T13:53:59Z] [b467f460] [dns] - static: resolved record: 3.225.200.224
[DEBUG] [2021-05-04T13:53:59Z] [b467f460] [dns] - static: resolved record: 54.145.247.28
[DEBUG] [2021-05-04T13:53:59Z] [b467f460] [dns] - static: resolved record: 54.84.18.235
[DEBUG] [2021-05-04T13:53:59Z] [b467f460] [dns] - static: resolved record: 18.207.61.253
[DEBUG] [2021-05-04T13:53:59Z] [b467f460] [dns] - static: resolved record: 52.21.198.210
[DEBUG] [2021-05-04T13:53:59Z] [b467f460] [dns] - static: resolved record: 54.147.70.168
[DEBUG] [2021-05-04T13:53:59Z] [b467f460] [dns] - static: resolved record: 2406:da00:ff00::23ad:4123
[DEBUG] [2021-05-04T13:53:59Z] [b467f460] [dns] - static: resolved record: 2406:da00:ff00::22cd:842b
[DEBUG] [2021-05-04T13:53:59Z] [b467f460] [dns] - static: resolved record: 2406:da00:ff00::3db:a390
[DEBUG] [2021-05-04T13:53:59Z] [b467f460] [dns] - static: resolved record: 2406:da00:ff00::34c8:5c29
[DEBUG] [2021-05-04T13:53:59Z] [b467f460] [dns] - static: resolved record: 2406:da00:ff00::36a3:c4c1
[DEBUG] [2021-05-04T13:53:59Z] [b467f460] [dns] - static: resolved record: 2406:da00:ff00::3e1:b950
[DEBUG] [2021-05-04T13:53:59Z] [b467f460] [dns] - static: resolved record: 2406:da00:ff00::3e1:f981
[DEBUG] [2021-05-04T13:53:59Z] [b467f460] [dns] - static: resolved record: 2406:da00:ff00::3417:2507
[DEBUG] [2021-05-04T13:53:59Z] [b467f460] [dns] - static, resolving host <my_endpoint>-ats.iot.us-east-1.amazonaws.com successful, returned 16 addresses
[DEBUG] [2021-05-04T13:53:59Z] [b51fe460] [tls-handler] - id=0xb48175d0: Alpn protocol negotiated as x-amzn-mqtt-ca
[DEBUG] [2021-05-04T13:53:59Z] [b51fe460] [tls-handler] - id=0xb48175d0: Remote server name is <my_endpoint>-ats.iot.us-east-1.amazonaws.com
[DEBUG] [2021-05-04T13:53:59Z] [b51fe460] [channel-bootstrap] - id=0x164f998: tls negotiation result 0 on channel 0xb4806838
[DEBUG] [2021-05-04T13:53:59Z] [b51fe460] [mqtt-client] - id=0x163d9b8: Connection successfully opened, sending CONNECT packet
[DEBUG] [2021-05-04T13:53:59Z] [b51fe460] [task-scheduler] - id=0xb48353c8: Scheduling mqtt_connack_timeout task for future execution at time 2079454727914
[DEBUG] [2021-05-04T13:53:59Z] [b51fe460] [mqtt-client] - id=0x163d9b8: Adding username ?SDK=PythonV2&Version=dev to connection
[DEBUG] [2021-05-04T13:53:59Z] [b51fe460] [task-scheduler] - id=0xb4805260: Scheduling socket_written_task task for immediate execution
[DEBUG] [2021-05-04T13:53:59Z] [b51fe460] [task-scheduler] - id=0xb4805260: Running socket_written_task task with <Running> status
[DEBUG] [2021-05-04T13:53:59Z] [b51fe460] [mqtt-client] - id=0x163d9b8: connection was accepted, switch state from 2 to CONNECTED.
[DEBUG] [2021-05-04T13:53:59Z] [b51fe460] [task-scheduler] - id=0x1715368: Scheduling mqtt_outgoing_request_task task for immediate execution
aws-iotsdk> [2021-05-04 13:53:59.568730] [aws] Connection resumed. return_code: ConnectReturnCode.ACCEPTED session_present: True
[DEBUG] [2021-05-04T13:53:59Z] [b51fe460] [task-scheduler] - id=0x163dac8: Scheduling mqtt_ping task for future execution at time 2106715588247
[DEBUG] [2021-05-04T13:53:59Z] [b51fe460] [task-scheduler] - id=0x1715368: Running mqtt_outgoing_request_task task with <Running> status
[DEBUG] [2021-05-04T13:53:59Z] [b51fe460] [task-scheduler] - id=0xb4805260: Scheduling socket_written_task task for immediate execution
[DEBUG] [2021-05-04T13:53:59Z] [b51fe460] [task-scheduler] - id=0xb4805260: Running socket_written_task task with <Running> status
[DEBUG] [2021-05-04T13:53:59Z] [b51fe460] [task-scheduler] - id=0xb4805260: Scheduling socket_written_task task for immediate execution
[DEBUG] [2021-05-04T13:53:59Z] [b51fe460] [task-scheduler] - id=0xb4805260: Running socket_written_task task with <Running> status
[DEBUG] [2021-05-04T13:53:59Z] [b51fe460] [task-scheduler] - id=0xb4805260: Scheduling socket_written_task task for immediate execution
[DEBUG] [2021-05-04T13:53:59Z] [b51fe460] [task-scheduler] - id=0xb4805260: Running socket_written_task task with <Running> status
[DEBUG] [2021-05-04T13:54:00Z] [b467f460] [dns] - static: resolving host <my_endpoint>-ats.iot.us-east-1.amazonaws.com
[DEBUG] [2021-05-04T13:54:00Z] [b467f460] [dns] - static: resolved record: 54.147.70.168
[DEBUG] [2021-05-04T13:54:00Z] [b467f460] [dns] - static: resolved record: 34.206.245.43
[DEBUG] [2021-05-04T13:54:00Z] [b467f460] [dns] - static: resolved record: 54.158.175.156
[DEBUG] [2021-05-04T13:54:00Z] [b467f460] [dns] - static: resolved record: 3.225.200.224
[DEBUG] [2021-05-04T13:54:00Z] [b467f460] [dns] - static: resolved record: 54.145.247.28
[DEBUG] [2021-05-04T13:54:00Z] [b467f460] [dns] - static: resolved record: 54.84.18.235
[DEBUG] [2021-05-04T13:54:00Z] [b467f460] [dns] - static: resolved record: 18.207.61.253
[DEBUG] [2021-05-04T13:54:00Z] [b467f460] [dns] - static: resolved record: 52.21.198.210
[DEBUG] [2021-05-04T13:54:00Z] [b467f460] [dns] - static: resolved record: 2406:da00:ff00::3417:2507
[DEBUG] [2021-05-04T13:54:00Z] [b467f460] [dns] - static: resolved record: 2406:da00:ff00::23ad:4123
[DEBUG] [2021-05-04T13:54:00Z] [b467f460] [dns] - static: resolved record: 2406:da00:ff00::22cd:842b
[DEBUG] [2021-05-04T13:54:00Z] [b467f460] [dns] - static: resolved record: 2406:da00:ff00::3db:a390
[DEBUG] [2021-05-04T13:54:00Z] [b467f460] [dns] - static: resolved record: 2406:da00:ff00::34c8:5c29
[DEBUG] [2021-05-04T13:54:00Z] [b467f460] [dns] - static: resolved record: 2406:da00:ff00::36a3:c4c1
[DEBUG] [2021-05-04T13:54:00Z] [b467f460] [dns] - static: resolved record: 2406:da00:ff00::3e1:b950
[DEBUG] [2021-05-04T13:54:00Z] [b467f460] [dns] - static: resolved record: 2406:da00:ff00::3e1:f981
[DEBUG] [2021-05-04T13:54:00Z] [b467f460] [dns] - static, resolving host <my_endpoint>-ats.iot.us-east-1.amazonaws.com successful, returned 16 addresses
[DEBUG] [2021-05-04T13:54:00Z] [b51fe460] [tls-handler] - id=0xb48175d0: Alert code 0
[DEBUG] [2021-05-04T13:54:00Z] [b51fe460] [task-scheduler] - id=0xb4806978: Scheduling channel_shutdown task for immediate execution
[DEBUG] [2021-05-04T13:54:00Z] [b51fe460] [task-scheduler] - id=0xb4806978: Running channel_shutdown task with <Running> status
[DEBUG] [2021-05-04T13:54:00Z] [b51fe460] [channel] - id=0xb4806838: beginning shutdown process
[DEBUG] [2021-05-04T13:54:00Z] [b51fe460] [channel] - id=0xb4806838: handler 0xb482ec00 shutdown in read dir completed.
[DEBUG] [2021-05-04T13:54:00Z] [b51fe460] [tls-handler] - id=0xb48175d0: Shutting down read direction with error code 0
[DEBUG] [2021-05-04T13:54:00Z] [b51fe460] [channel] - id=0xb4806838: handler 0xb48175d0 shutdown in read dir completed.
[DEBUG] [2021-05-04T13:54:00Z] [b51fe460] [channel] - id=0xb4806838: handler 0x163d9cc shutdown in read dir completed.
[DEBUG] [2021-05-04T13:54:00Z] [b51fe460] [task-scheduler] - id=0xb4806850: Scheduling (null) task for immediate execution
[DEBUG] [2021-05-04T13:54:00Z] [b51fe460] [task-scheduler] - id=0xb4806850: Running (null) task with <Running> status
[INFO] [2021-05-04T13:54:00Z] [b51fe460] [mqtt-client] - id=0x163d9b8: sending disconnect message as part of graceful shutdown.
[DEBUG] [2021-05-04T13:54:00Z] [b51fe460] [mqtt-client] - id=0x163d9b8: failed to send courteous disconnect io message
[DEBUG] [2021-05-04T13:54:00Z] [b51fe460] [channel] - id=0xb4806838: handler 0x163d9cc shutdown in write dir completed.
[DEBUG] [2021-05-04T13:54:00Z] [b51fe460] [tls-handler] - id=0xb48175d0: Scheduling delayed write direction shutdown
[DEBUG] [2021-05-04T13:54:00Z] [b51fe460] [task-scheduler] - id=0xb48176e0: Scheduling s2n_delayed_shutdown task for future execution at time 2077377824580
[DEBUG] [2021-05-04T13:54:00Z] [b51fe460] [task-scheduler] - id=0xb48176e0: Running s2n_delayed_shutdown task with <Running> status
[DEBUG] [2021-05-04T13:54:00Z] [b51fe460] [tls-handler] - id=0xb48175d0: Delayed shut down in write direction
[DEBUG] [2021-05-04T13:54:00Z] [b51fe460] [task-scheduler] - id=0xb4805260: Scheduling socket_written_task task for immediate execution
[DEBUG] [2021-05-04T13:54:00Z] [b51fe460] [channel] - id=0xb4806838: handler 0xb48175d0 shutdown in write dir completed.
[DEBUG] [2021-05-04T13:54:00Z] [b51fe460] [socket] - id=0xb4805118 fd=10: closing
[DEBUG] [2021-05-04T13:54:00Z] [b51fe460] [task-scheduler] - id=0xb482b970: Scheduling epoll_event_loop_unsubscribe_cleanup task for immediate execution
[DEBUG] [2021-05-04T13:54:00Z] [b51fe460] [task-scheduler] - id=0xb4805260: Running socket_written_task task with <Canceled> status
[DEBUG] [2021-05-04T13:54:00Z] [b51fe460] [task-scheduler] - id=0xb482ec60: Scheduling socket_handler_close task for immediate execution
[DEBUG] [2021-05-04T13:54:00Z] [b51fe460] [task-scheduler] - id=0xb482b970: Running epoll_event_loop_unsubscribe_cleanup task with <Running> status
[DEBUG] [2021-05-04T13:54:00Z] [b51fe460] [task-scheduler] - id=0xb482ec60: Running socket_handler_close task with <Running> status
[DEBUG] [2021-05-04T13:54:00Z] [b51fe460] [channel] - id=0xb4806838: handler 0xb482ec00 shutdown in write dir completed.
[DEBUG] [2021-05-04T13:54:00Z] [b51fe460] [task-scheduler] - id=0xb4806850: Scheduling (null) task for immediate execution
[DEBUG] [2021-05-04T13:54:00Z] [b51fe460] [task-scheduler] - id=0xb4806850: Running (null) task with <Running> status
[DEBUG] [2021-05-04T13:54:00Z] [b51fe460] [channel] - id=0xb4806838: during shutdown, canceling task 0xb48175e8
[DEBUG] [2021-05-04T13:54:00Z] [b51fe460] [task-scheduler] - id=0xb48175e8: Running tls_timeout task with <Canceled> status
[DEBUG] [2021-05-04T13:54:00Z] [b51fe460] [channel] - id=0xb4806838: during shutdown, canceling task 0xb48353c8
[DEBUG] [2021-05-04T13:54:00Z] [b51fe460] [task-scheduler] - id=0xb48353c8: Running mqtt_connack_timeout task with <Canceled> status
[DEBUG] [2021-05-04T13:54:00Z] [b51fe460] [channel] - id=0xb4806838: during shutdown, canceling task 0x163dac8
[DEBUG] [2021-05-04T13:54:00Z] [b51fe460] [task-scheduler] - id=0x163dac8: Running mqtt_ping task with <Canceled> status
[DEBUG] [2021-05-04T13:54:00Z] [b51fe460] [channel-bootstrap] - id=0x164f998: channel 0xb4806838 shutdown with error 0.
[DEBUG] [2021-05-04T13:54:00Z] [b51fe460] [mqtt-client] - id=0x163d9b8: connection was unexpected interrupted, switch state to RECONNECTING.
[DEBUG] [2021-05-04T13:54:00Z] [b51fe460] [mqtt-client] - id=0x163d9b8: Connection interrupted, calling callback and attempting reconnect
aws-iotsdk> [2021-05-04 13:54:00.265671] [aws] Connection interrupted. error: AWS_ERROR_MQTT_UNEXPECTED_HANGUP: The connection was closed unexpectedly.
[DEBUG] [2021-05-04T13:54:00Z] [b51fe460] [dns] - id=0x162d7a0: Host resolution requested for <my_endpoint>-ats.iot.us-east-1.amazonaws.com
[DEBUG] [2021-05-04T13:54:00Z] [b51fe460] [dns] - id=0x162d7a0: cached entries found for <my_endpoint>-ats.iot.us-east-1.amazonaws.com returning to caller.
[DEBUG] [2021-05-04T13:54:00Z] [b51fe460] [task-scheduler] - id=0xb482fc00: Scheduling attempt_connection task for immediate execution
[DEBUG] [2021-05-04T13:54:00Z] [b51fe460] [task-scheduler] - id=0xb482f5e0: Scheduling attempt_connection task for immediate execution
[DEBUG] [2021-05-04T13:54:00Z] [b51fe460] [channel] - id=0xb4806838: destroying channel.
[DEBUG] [2021-05-04T13:54:00Z] [b51fe460] [channel-bootstrap] - id=0x164f998: releasing bootstrap reference
[DEBUG] [2021-05-04T13:54:00Z] [b51fe460] [task-scheduler] - id=0xb482fc00: Running attempt_connection task with <Running> status
[DEBUG] [2021-05-04T13:54:00Z] [b51fe460] [socket] - id=0xb4805118 fd=10: initializing with domain 1 and type 0
[DEBUG] [2021-05-04T13:54:00Z] [b51fe460] [socket] - id=0xb4805118 fd=10: setting socket options to: keep-alive 0, keep idle 0, keep-alive interval 0, keep-alive probe count 0.
[WARN] [2021-05-04T13:54:00Z] [b51fe460] [socket] - id=0xb4805118 fd=10: setsockopt() for NO_SIGNAL failed with errno 92. If you are having SIGPIPE signals thrown, you may want to install a signal trap in your application layer.
[DEBUG] [2021-05-04T13:54:00Z] [b51fe460] [socket] - id=0xb4805118 fd=10: beginning connect.
[DEBUG] [2021-05-04T13:54:00Z] [b51fe460] [socket] - id=0xb4805118 fd=10: connecting to endpoint 2406:da00:ff00::3417:2507:443.
[ERROR] [2021-05-04T13:54:00Z] [b51fe460] [socket] - id=0xb4805118 fd=10: connect failed with error code 101.
[INFO] [2021-05-04T13:54:00Z] [b51fe460] [dns] - id=0x162d7a0: recording failure for record 2406:da00:ff00::3417:2507 for <my_endpoint>-ats.iot.us-east-1.amazonaws.com, moving to bad list
[DEBUG] [2021-05-04T13:54:00Z] [b51fe460] [dns] - static: purging address 2406:da00:ff00::3417:2507 for host <my_endpoint>-ats.iot.us-east-1.amazonaws.com from the cache due to cache eviction or shutdown
[DEBUG] [2021-05-04T13:54:00Z] [b51fe460] [socket] - id=0xb4805118 fd=10: is still open, closing...
[DEBUG] [2021-05-04T13:54:00Z] [b51fe460] [socket] - id=0xb4805118 fd=10: closing
[ERROR] [2021-05-04T13:54:00Z] [b51fe460] [channel-bootstrap] - id=0x164f998: failed to create socket with error 1049
[DEBUG] [2021-05-04T13:54:00Z] [b51fe460] [task-scheduler] - id=0xb482f5e0: Running attempt_connection task with <Running> status
[DEBUG] [2021-05-04T13:54:00Z] [b51fe460] [socket] - id=0xb4805118 fd=10: initializing with domain 0 and type 0
[DEBUG] [2021-05-04T13:54:00Z] [b51fe460] [socket] - id=0xb4805118 fd=10: setting socket options to: keep-alive 0, keep idle 0, keep-alive interval 0, keep-alive probe count 0.
[WARN] [2021-05-04T13:54:00Z] [b51fe460] [socket] - id=0xb4805118 fd=10: setsockopt() for NO_SIGNAL failed with errno 92. If you are having SIGPIPE signals thrown, you may want to install a signal trap in your application layer.
[DEBUG] [2021-05-04T13:54:00Z] [b51fe460] [socket] - id=0xb4805118 fd=10: beginning connect.
[DEBUG] [2021-05-04T13:54:00Z] [b51fe460] [socket] - id=0xb4805118 fd=10: connecting to endpoint 54.147.70.168:443.
[DEBUG] [2021-05-04T13:54:00Z] [b51fe460] [task-scheduler] - id=0xb4803ea8: Scheduling (null) task for future execution at time 2082467830580
[DEBUG] [2021-05-04T13:54:00Z] [b51fe460] [task-scheduler] - id=0xb4805680: Scheduling epoll_event_loop_unsubscribe_cleanup task for immediate execution
[INFO] [2021-05-04T13:54:00Z] [b51fe460] [socket] - id=0xb4805118 fd=10: connection success
[DEBUG] [2021-05-04T13:54:00Z] [b51fe460] [socket] - id=0xb4805118 fd=10: local endpoint 10.42.0.177:50080
[DEBUG] [2021-05-04T13:54:00Z] [b51fe460] [socket] - id=0xb4805118 fd=10: assigning to event loop 0x1684fc8
[DEBUG] [2021-05-04T13:54:00Z] [b51fe460] [channel-bootstrap] - id=0x164f998: client connection on socket 0xb4805118 completed with error 0.
[DEBUG] [2021-05-04T13:54:00Z] [b51fe460] [channel] - id=0xb4806838: Beginning creation and setup of new channel.
[DEBUG] [2021-05-04T13:54:00Z] [b51fe460] [task-scheduler] - id=0xb4835560: Scheduling on_channel_setup_complete task for immediate execution
[DEBUG] [2021-05-04T13:54:00Z] [b51fe460] [task-scheduler] - id=0xb4805680: Running epoll_event_loop_unsubscribe_cleanup task with <Running> status
[DEBUG] [2021-05-04T13:54:00Z] [b51fe460] [task-scheduler] - id=0xb4835560: Running on_channel_setup_complete task with <Running> status
[DEBUG] [2021-05-04T13:54:00Z] [b51fe460] [channel] - id=0xb4806838: setup complete, notifying caller.
[DEBUG] [2021-05-04T13:54:00Z] [b51fe460] [channel] - id=0xb4806838: message pool 0xb4805c40 found in event-loop local storage: using it.
[DEBUG] [2021-05-04T13:54:00Z] [b51fe460] [channel-bootstrap] - id=0x164f998: channel 0xb4806838 setup succeeded: bootstrapping.
[DEBUG] [2021-05-04T13:54:00Z] [b51fe460] [socket-handler] - id=0xb482b660: Socket handler created with max_read_size of 16384
[DEBUG] [2021-05-04T13:54:00Z] [b51fe460] [task-scheduler] - id=0xb48175e8: Scheduling tls_timeout task for future execution at time 2087638712914
[DEBUG] [2021-05-04T13:54:00Z] [b51fe460] [task-scheduler] - id=0xb4805260: Scheduling socket_written_task task for immediate execution
[DEBUG] [2021-05-04T13:54:00Z] [b51fe460] [task-scheduler] - id=0xb4805260: Running socket_written_task task with <Running> status
[DEBUG] [2021-05-04T13:54:00Z] [b51fe460] [task-scheduler] - id=0xb4805260: Scheduling socket_written_task task for immediate execution
[DEBUG] [2021-05-04T13:54:00Z] [b51fe460] [task-scheduler] - id=0xb4805260: Running socket_written_task task with <Running> status
[DEBUG] [2021-05-04T13:54:01Z] [b467f460] [dns] - static: resolving host <my_endpoint>-ats.iot.us-east-1.amazonaws.com
[DEBUG] [2021-05-04T13:54:01Z] [b467f460] [dns] - static: resolved record: 52.21.198.210
[DEBUG] [2021-05-04T13:54:01Z] [b467f460] [dns] - static: resolved record: 54.147.70.168
[DEBUG] [2021-05-04T13:54:01Z] [b467f460] [dns] - static: resolved record: 34.206.245.43
[DEBUG] [2021-05-04T13:54:01Z] [b467f460] [dns] - static: resolved record: 54.158.175.156
[DEBUG] [2021-05-04T13:54:01Z] [b467f460] [dns] - static: resolved record: 3.225.200.224
[DEBUG] [2021-05-04T13:54:01Z] [b51fe460] [tls-handler] - id=0xb48175d0: Alpn protocol negotiated as x-amzn-mqtt-ca
[DEBUG] [2021-05-04T13:54:01Z] [b51fe460] [tls-handler] - id=0xb48175d0: Remote server name is <my_endpoint>-ats.iot.us-east-1.amazonaws.com
[DEBUG] [2021-05-04T13:54:01Z] [b51fe460] [channel-bootstrap] - id=0x164f998: tls negotiation result 0 on channel 0xb4806838
[DEBUG] [2021-05-04T13:54:01Z] [b51fe460] [mqtt-client] - id=0x163d9b8: Connection successfully opened, sending CONNECT packet
[DEBUG] [2021-05-04T13:54:01Z] [b51fe460] [task-scheduler] - id=0xb482d770: Scheduling mqtt_connack_timeout task for future execution at time 2081369652914
[DEBUG] [2021-05-04T13:54:01Z] [b51fe460] [mqtt-client] - id=0x163d9b8: Adding username ?SDK=PythonV2&Version=dev to connection
[DEBUG] [2021-05-04T13:54:01Z] [b51fe460] [task-scheduler] - id=0xb4805260: Scheduling socket_written_task task for immediate execution
[DEBUG] [2021-05-04T13:54:01Z] [b467f460] [dns] - static: resolved record: 54.145.247.28
[DEBUG] [2021-05-04T13:54:01Z] [b467f460] [dns] - static: resolved record: 54.84.18.235
[DEBUG] [2021-05-04T13:54:01Z] [b467f460] [dns] - static: resolved record: 18.207.61.253
[DEBUG] [2021-05-04T13:54:01Z] [b467f460] [dns] - static: resolved record: 2406:da00:ff00::3e1:f981
[DEBUG] [2021-05-04T13:54:01Z] [b51fe460] [task-scheduler] - id=0xb4805260: Running socket_written_task task with <Running> status
[DEBUG] [2021-05-04T13:54:01Z] [b467f460] [dns] - static: resolved record: 2406:da00:ff00::3417:2507
[DEBUG] [2021-05-04T13:54:01Z] [b467f460] [dns] - static: resolved record: 2406:da00:ff00::23ad:4123
[DEBUG] [2021-05-04T13:54:01Z] [b467f460] [dns] - static: resolved record: 2406:da00:ff00::22cd:842b
[DEBUG] [2021-05-04T13:54:01Z] [b467f460] [dns] - static: resolved record: 2406:da00:ff00::3db:a390
[DEBUG] [2021-05-04T13:54:01Z] [b467f460] [dns] - static: resolved record: 2406:da00:ff00::34c8:5c29
[DEBUG] [2021-05-04T13:54:01Z] [b467f460] [dns] - static: resolved record: 2406:da00:ff00::36a3:c4c1
[DEBUG] [2021-05-04T13:54:01Z] [b467f460] [dns] - static: resolved record: 2406:da00:ff00::3e1:b950
[DEBUG] [2021-05-04T13:54:01Z] [b467f460] [dns] - static, resolving host <my_endpoint>-ats.iot.us-east-1.amazonaws.com successful, returned 16 addresses
aws-iotsdk> [2021-05-04 13:54:01.464140] [aws] Connection resumed. return_code: ConnectReturnCode.ACCEPTED session_present: True
[DEBUG] [2021-05-04T13:54:01Z] [b51fe460] [mqtt-client] - id=0x163d9b8: connection was accepted, switch state from 2 to CONNECTED.
[DEBUG] [2021-05-04T13:54:01Z] [b51fe460] [task-scheduler] - id=0x1715368: Scheduling mqtt_outgoing_request_task task for immediate execution
[DEBUG] [2021-05-04T13:54:01Z] [b51fe460] [task-scheduler] - id=0x163dac8: Scheduling mqtt_ping task for future execution at time 2108610288247
[DEBUG] [2021-05-04T13:54:01Z] [b51fe460] [task-scheduler] - id=0x1715368: Running mqtt_outgoing_request_task task with <Running> status
[DEBUG] [2021-05-04T13:54:01Z] [b51fe460] [task-scheduler] - id=0xb4805260: Scheduling socket_written_task task for immediate execution
[DEBUG] [2021-05-04T13:54:01Z] [b51fe460] [task-scheduler] - id=0xb4805260: Running socket_written_task task with <Running> status
[DEBUG] [2021-05-04T13:54:01Z] [b51fe460] [task-scheduler] - id=0xb4805260: Scheduling socket_written_task task for immediate execution
[DEBUG] [2021-05-04T13:54:01Z] [b51fe460] [task-scheduler] - id=0xb4805260: Running socket_written_task task with <Running> status
[DEBUG] [2021-05-04T13:54:01Z] [b51fe460] [task-scheduler] - id=0xb4805260: Scheduling socket_written_task task for immediate execution
[DEBUG] [2021-05-04T13:54:01Z] [b51fe460] [task-scheduler] - id=0xb4805260: Running socket_written_task task with <Running> status
[DEBUG] [2021-05-04T13:54:01Z] [b51fe460] [task-scheduler] - id=0xb4805260: Scheduling socket_written_task task for immediate execution
[DEBUG] [2021-05-04T13:54:01Z] [b51fe460] [task-scheduler] - id=0xb4805260: Running socket_written_task task with <Running> status
[DEBUG] [2021-05-04T13:54:02Z] [b51fe460] [tls-handler] - id=0xb48175d0: Alert code 0
[DEBUG] [2021-05-04T13:54:02Z] [b51fe460] [task-scheduler] - id=0xb4806978: Scheduling channel_shutdown task for immediate execution
[DEBUG] [2021-05-04T13:54:02Z] [b51fe460] [task-scheduler] - id=0xb4806978: Running channel_shutdown task with <Running> status
[DEBUG] [2021-05-04T13:54:02Z] [b51fe460] [channel] - id=0xb4806838: beginning shutdown process
[DEBUG] [2021-05-04T13:54:02Z] [b51fe460] [channel] - id=0xb4806838: handler 0xb482b660 shutdown in read dir completed.
[DEBUG] [2021-05-04T13:54:02Z] [b51fe460] [tls-handler] - id=0xb48175d0: Shutting down read direction with error code 0
[DEBUG] [2021-05-04T13:54:02Z] [b51fe460] [channel] - id=0xb4806838: handler 0xb48175d0 shutdown in read dir completed.
[DEBUG] [2021-05-04T13:54:02Z] [b51fe460] [channel] - id=0xb4806838: handler 0x163d9cc shutdown in read dir completed.
[DEBUG] [2021-05-04T13:54:02Z] [b51fe460] [task-scheduler] - id=0xb4806850: Scheduling (null) task for immediate execution
[DEBUG] [2021-05-04T13:54:02Z] [b51fe460] [task-scheduler] - id=0xb4806850: Running (null) task with <Running> status
[INFO] [2021-05-04T13:54:02Z] [b51fe460] [mqtt-client] - id=0x163d9b8: sending disconnect message as part of graceful shutdown.
[DEBUG] [2021-05-04T13:54:02Z] [b51fe460] [mqtt-client] - id=0x163d9b8: failed to send courteous disconnect io message
[DEBUG] [2021-05-04T13:54:02Z] [b51fe460] [channel] - id=0xb4806838: handler 0x163d9cc shutdown in write dir completed.
[DEBUG] [2021-05-04T13:54:02Z] [b51fe460] [tls-handler] - id=0xb48175d0: Scheduling delayed write direction shutdown
[DEBUG] [2021-05-04T13:54:02Z] [b51fe460] [task-scheduler] - id=0xb48176e0: Scheduling s2n_delayed_shutdown task for future execution at time 2079278874581
[DEBUG] [2021-05-04T13:54:02Z] [b51fe460] [task-scheduler] - id=0xb48176e0: Running s2n_delayed_shutdown task with <Running> status
[DEBUG] [2021-05-04T13:54:02Z] [b51fe460] [tls-handler] - id=0xb48175d0: Delayed shut down in write direction
[DEBUG] [2021-05-04T13:54:02Z] [b51fe460] [task-scheduler] - id=0xb4805260: Scheduling socket_written_task task for immediate execution
[DEBUG] [2021-05-04T13:54:02Z] [b51fe460] [channel] - id=0xb4806838: handler 0xb48175d0 shutdown in write dir completed.
[DEBUG] [2021-05-04T13:54:02Z] [b51fe460] [socket] - id=0xb4805118 fd=10: closing
[DEBUG] [2021-05-04T13:54:02Z] [b51fe460] [task-scheduler] - id=0xb48349b0: Scheduling epoll_event_loop_unsubscribe_cleanup task for immediate execution
[DEBUG] [2021-05-04T13:54:02Z] [b51fe460] [task-scheduler] - id=0xb4805260: Running socket_written_task task with <Canceled> status
[DEBUG] [2021-05-04T13:54:02Z] [b51fe460] [task-scheduler] - id=0xb482b6c0: Scheduling socket_handler_close task for immediate execution
[DEBUG] [2021-05-04T13:54:02Z] [b51fe460] [task-scheduler] - id=0xb48349b0: Running epoll_event_loop_unsubscribe_cleanup task with <Running> status
[DEBUG] [2021-05-04T13:54:02Z] [b51fe460] [task-scheduler] - id=0xb482b6c0: Running socket_handler_close task with <Running> status
[DEBUG] [2021-05-04T13:54:02Z] [b51fe460] [channel] - id=0xb4806838: handler 0xb482b660 shutdown in write dir completed.
[DEBUG] [2021-05-04T13:54:02Z] [b51fe460] [task-scheduler] - id=0xb4806850: Scheduling (null) task for immediate execution
[DEBUG] [2021-05-04T13:54:02Z] [b51fe460] [task-scheduler] - id=0xb4806850: Running (null) task with <Running> status
[DEBUG] [2021-05-04T13:54:02Z] [b51fe460] [channel] - id=0xb4806838: during shutdown, canceling task 0xb48175e8
[DEBUG] [2021-05-04T13:54:02Z] [b51fe460] [task-scheduler] - id=0xb48175e8: Running tls_timeout task with <Canceled> status
[DEBUG] [2021-05-04T13:54:02Z] [b51fe460] [channel] - id=0xb4806838: during shutdown, canceling task 0xb482d770
[DEBUG] [2021-05-04T13:54:02Z] [b51fe460] [task-scheduler] - id=0xb482d770: Running mqtt_connack_timeout task with <Canceled> status
[DEBUG] [2021-05-04T13:54:02Z] [b51fe460] [channel] - id=0xb4806838: during shutdown, canceling task 0x163dac8
[DEBUG] [2021-05-04T13:54:02Z] [b51fe460] [task-scheduler] - id=0x163dac8: Running mqtt_ping task with <Canceled> status
[DEBUG] [2021-05-04T13:54:02Z] [b51fe460] [channel-bootstrap] - id=0x164f998: channel 0xb4806838 shutdown with error 0.
[DEBUG] [2021-05-04T13:54:02Z] [b51fe460] [mqtt-client] - id=0x163d9b8: connection was unexpected interrupted, switch state to RECONNECTING.
[DEBUG] [2021-05-04T13:54:02Z] [b51fe460] [mqtt-client] - id=0x163d9b8: Connection interrupted, calling callback and attempting reconnect
aws-iotsdk> [2021-05-04 13:54:02.177324] [aws] Connection interrupted. error: AWS_ERROR_MQTT_UNEXPECTED_HANGUP: The connection was closed unexpectedly.
追加のコンテキスト
解決の試み：mqttビルダーのパラメーターを変更し、デバイスを別の証明書に登録しました
デバイスは、JITPプロセスを介して自動的に登録され、次のJTIP_PubSubポリシーが添付されます。
bug
qcabrol
👍3
全てのコメント8件
+1
Hadatko
2021年05月04日
もう少しコンテキストについて：
ビルドと構成がまったく同じ別のデバイスが正常に接続されています。 もう1つはまだそうではありません。 公開サイズの割り当てに達しているのではないかと思います。 まだ調査中です...その場合、私はエラーを予期します。
qcabrol
2021年05月04日
+ 1-フィールド内のデバイスの約4％でこれが見られます
awsiotsdk = 1.5.11およびawscrt = 0.11.8のarm64デバイスでUbuntu18.04を使用しています
JoelHensley
2021年05月04日
👍2
ビルドと構成がまったく同じ別のデバイスが正常に接続されています。 もう1つはまだそうではありません。
すべてのデバイスが一意のクライアントIDで接続していますよね？
2番目のデバイスが同じクライアントIDに接続する場合、サーバーはその新しい接続を受け入れ、古い接続を切断します。 これにより、最初のデバイスが再接続されて2番目のデバイスがキックオフされるため、2番目のデバイスが再接続されて最初のデバイスがキックオフされます。
私が話している「client-id」は、 mqtt_connection_builder関数またはawscrt.mqtt.Connectionクラスに渡されるclient_id引数です。
graebm
2021年05月05日
また、GBのデータを吸い込む高速再接続ループを吸い込むWOW。
私たちの実装では、再接続中に指数バックオフタイマーを使用しようとしますが、接続が「成功」したと見なされると、タイマーはゼロにリセットされます。 接続が成功した直後に何かが接続を切断した場合、そのタイマーがゼロになっているため、デバイスはすぐに再接続します。
タイマーをリセットする前に、接続が「安定」したままになるのをしばらく待つ必要があるようです...
graebm
2021年05月05日
👍1
サーバーがデバイスを切断している場合は、その理由を確認できます。 CloudWatchログを使用したAWSIoTのモニタリングを参照してください
graebm
2021年05月05日
こんにちは@graebm
ポインタをありがとう
残念ながら、AWSログのINFOレベルはここではあまり役に立ちません。
クライアントIDの一意性に関しては、アプリケーションがuuidモジュールからpython uuid4（）を使用して開始するたびに生成されます。
client_id = "auto-%s" % str(uuid.uuid4()).upper()
これは、非常に不運だったために重複したuuidがあったとしても、新しいクライアントuuidを使用するアプリケーションの再起動後も問題が発生しないことを意味します。 この可能性について少し詳しく説明します。
私が検討しているもう1つのことは、ある種の公開パケットサイズの割り当てに達しているかどうかです。 ネットワークの問題が原因でデバイスが切断された場合、全体的なデータ使用量を制限するために、1回の呼び出しで公開した多くのパケットをまとめます。 現在、その最大バンドルサイズを制限しようとしています。これが、動作に影響するかどうかを確認してください。 最大サイズは128KBです。
質問は、リクエストを送信する前に、SDKクライアント側にクォータ関連のフィルターがありますか？ （これにより、多くの非準拠パケットが防止され、ネットワークデータに接続して食べる前に、不正な要求が発生した場合のデバッグを容易にする明示的なエラーが発生します）。
qcabrol
2021年05月05日
つまり、結論は次のとおりです。
- パケットサイズを少し膨らませる方法で生のバイトをシリアル化します（後でバイナリパケットを使用します）
- 接続が失われると、バイト配列のグループ化されたリストでパケットを送信しました（最大1000個のメッセージがグループ化されました）
さまざまなシナリオで計算を行いましたが、この状況では割り当てを超えていました。
1つのパケットにバンドルされるメッセージの最大数を制限し、それが機能しました。
これは、デバイスがしばらくの間接続を失い、いくつかのアラートロギングが発生し、クォータがサポートしている制限を超えるデータバーストが作成されたために発生しました。
私見ですが、SDKは、パケットを送信する前に、ハードセットクォータ違反がないかパケットをチェックし、エラーをスローして、そのような状況のデバッグを容易にする必要があります。 ここでは、出典は不明でした。
サイズが128kBを超えるパケットをtryループで公開することにより、状況を再現できるはずです。
qcabrol
2021年05月05日
このページは役に立ちましたか？
0 / 5 - 0 評価
関連する問題
satay99
·
6コメント
shravan097
·
6コメント
mkozjak
·
17コメント
mkozjak
·
8コメント
banuprathap
·
10コメント
最も参考になるコメント
+ 1-フィールド内のデバイスの約4％でこれが見られます
awsiotsdk = 1.5.11およびawscrt = 0.11.8のarm64デバイスでUbuntu18.04を使用しています