Aws-iot-device-sdk-python-v2: message='The connection was closed unexpectedly.', code=5134 region us-west-1

What version of python3 and Ubuntu are you using? Also what permissions are you using for the IAM policy? It looks like you are having a problem with connecting, so I recommend starting with AWSIoTFullAccess to rule out your policy being the problem. Just make sure to restrict it to only the actions and resources that you actually need later.

AWSIoTFullAccess:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "iot:*"
            ],
            "Resource": "*"
        }
    ]
}

As for the additional content you mentioned one of them has iot in the url while the other has iotv2. Which one is the one that is working for you?

Thanks for the response jmklix.
What you suggested is specifically for MQTT ?
I got websockets to work but only with .aws credentials which for our use case I don't think is desired.
Can websockets authenticate and authorize using the certificate files (xxxx.pem xxx.private.key xxx.public.key) that mqtt consumes?

Yes I caught on to the url iot/iot2 ... is there some documentation that explains the difference between the two iot core versions?
There are also 2 client python sdks afaik - one is for python 2.7 the other for python 3, are there more differences ? are they tightly paired with the iot & iot2 core versions ? Are they cross or back compatible ? Is there documentation for this ?

Yes, it is for MQTT. No you can't use cert files with web-sockets

You can edit the policy for IoT things that are already created on the AWS console. Select the thing and select security on the sub-side bar. There you should see certificates and you'll want to select that certificate. Once that is open select policy on the sub-side bar. Then select that policy and finally click Edit policy document and paste the AWSIoTFullAccess I listed previously. The default policy doesn't work with the v2 sdk's. Please let me know if you need any additional explanation.

The iot url is the one you want to use as the iotv2 should no longer be visible and will be removed shortly. The python and iot sdk's are independent of each other and have no relation. The iot v1 supports python 2 and 3 while the iot v2 only supports python 3. The iot sdk's are not cross or back compatible and documentation for the v2 api can be found here.

I have the same issue, I used the policy @jmklix included above, created a new certificate manually, linked it to the IoT Thing and the Policy and downloaded the .pem, private key and root-CA like this:

python3 shadow.py --endpoint xxxxxxxxxx-ats.iot.us-east-2.amazonaws.com --root-ca root-CA.crt --cert 726290b105-certificate.pem.crt --key 726290b105-private.pem.key --thing-name MyThingName

The shadow.py example says that --cert should be in .pem format, so I also tried with 726290b105-certificate.pem

Same error happens with pubsub.py example.

I am using a Raspberry pi with Python 3.

Tried installing with PyPI and installing with source.

The error happens in connected_future.result() in the shadow.py example.

I have a component which publishes using MQTT similar to the ipc_greengrass.py sample and it works well.

My suspicion is I am doing something wrong when including the key, cert and --rood-ca. Alghough I double checked the formats of the filres. I am getting this error:

Traceback (most recent call last):
  File "shadow.py", line 275, in <module>
    connected_future.result()
  File "/usr/lib/python3.7/concurrent/futures/_base.py", line 432, in result
    return self.__get_result()
  File "/usr/lib/python3.7/concurrent/futures/_base.py", line 384, in __get_result
    raise self._exception
awscrt.exceptions.AwsCrtError: AwsCrtError(name='AWS_ERROR_MQTT_UNEXPECTED_HANGUP', 
message='The connection was closed unexpectedly.',
code=5134)

Running with --verbosity --debug

[INFO] [2021-02-18T19:57:17Z] [b7b6bec0] [event-loop] - id=0xa4371f0: Initializing edge-triggered epoll
[DEBUG] [2021-02-18T19:57:17Z] [b7b6bec0] [event-loop] - id=0xa4371f0: Eventfd not available, falling back to pipe for cross-thread notification.
[INFO] [2021-02-18T19:57:17Z] [b7b6bec0] [event-loop] - id=0xa4371f0: Starting event-loop thread.
[INFO] [2021-02-18T19:57:17Z] [b7b6bec0] [dns] - id=0xa42b310: Initializing default host resolver with 16 max host entries.
[INFO] [2021-02-18T19:57:17Z] [b7050b40] [thread] - id=0xa3eeb8c: a cpu affinity was specified when launching this thread and set_mempolicy() is available on this system. Setting the memory policy to MPOL_PREFERRED
[WARN] [2021-02-18T19:57:17Z] [b7050b40] [thread] - id=0xa3eeb8c: call to set_mempolicy() failed with errno 38
[INFO] [2021-02-18T19:57:17Z] [b7b6bec0] [channel-bootstrap] - id=0xa44d200: Initializing client bootstrap with event-loop group 0xa43ea60
[INFO] [2021-02-18T19:57:17Z] [b684fb40] [event-loop] - id=0xa4371f0: main loop started
[INFO] [2021-02-18T19:57:17Z] [b684fb40] [event-loop] - id=0xa4371f0: default timeout 100000, and max events to process per tick 100
[DEBUG] [2021-02-18T19:57:17Z] [b7b6bec0] [tls-handler] - ctx: Certificate and key have been set, setting them up now.
[DEBUG] [2021-02-18T19:57:17Z] [b7b6bec0] [tls-handler] - ctx: Setting ALPN list x-amzn-mqtt-ca
Connecting to <endpoint> with client ID 'test-1e0e6e8b-ee1f-441d-8956-69aaedcb937d'...
[DEBUG] [2021-02-18T19:57:17Z] [b7b6bec0] [mqtt-client] - client=0xa4ee030: Initalizing MQTT client
[DEBUG] [2021-02-18T19:57:17Z] [b7b6bec0] [mqtt-client] - id=0xa450e40: Creating new connection
[DEBUG] [2021-02-18T19:57:17Z] [b7b6bec0] [mqtt-topic-tree] - tree=0xa450f94: Creating new topic tree
[DEBUG] [2021-02-18T19:57:17Z] [b7b6bec0] [mqtt-client] - id=0xa450e40: Begin connecting process, switch state to CONNECTING.
[INFO] [2021-02-18T19:57:17Z] [b7b6bec0] [mqtt-client] - id=0xa450e40: using ping timeout of 3000000000 ns
[DEBUG] [2021-02-18T19:57:17Z] [b7b6bec0] [dns] - id=0xa42b310: Host resolution requested for a1ypu7uyt3br9h-ats.iot.us-east-2.amazonaws.com
[DEBUG] [2021-02-18T19:57:17Z] [b7b6bec0] [dns] - id=0xa42b310: No cached entries found for a1ypu7uyt3br9h-ats.iot.us-east-2.amazonaws.com starting new resolver thread.
[DEBUG] [2021-02-18T19:57:17Z] [b5754b40] [dns] - static: resolving host a1ypu7uyt3br9h-ats.iot.us-east-2.amazonaws.com
[DEBUG] [2021-02-18T19:57:17Z] [b5754b40] [dns] - static: resolved record: 3.131.31.52
[DEBUG] [2021-02-18T19:57:17Z] [b5754b40] [dns] - static: resolved record: 52.15.119.124
[DEBUG] [2021-02-18T19:57:17Z] [b5754b40] [dns] - static: resolved record: 13.59.30.89
[DEBUG] [2021-02-18T19:57:17Z] [b5754b40] [dns] - static: resolved record: 2600:1f00:6000::383:1f34
[DEBUG] [2021-02-18T19:57:17Z] [b5754b40] [dns] - static: resolved record: 2600:1f00:6000::340f:777c
[DEBUG] [2021-02-18T19:57:17Z] [b5754b40] [dns] - static: resolved record: 2600:1f00:6000::d3b:1e59
[DEBUG] [2021-02-18T19:57:17Z] [b5754b40] [dns] - static: new address resolved 3.131.31.52 for host a1ypu7uyt3br9h-ats.iot.us-east-2.amazonaws.com caching
[DEBUG] [2021-02-18T19:57:17Z] [b5754b40] [dns] - static: new address resolved 52.15.119.124 for host a1ypu7uyt3br9h-ats.iot.us-east-2.amazonaws.com caching
[DEBUG] [2021-02-18T19:57:17Z] [b5754b40] [dns] - static: new address resolved 13.59.30.89 for host a1ypu7uyt3br9h-ats.iot.us-east-2.amazonaws.com caching
[DEBUG] [2021-02-18T19:57:17Z] [b5754b40] [dns] - static: new address resolved 2600:1f00:6000::383:1f34 for host a1ypu7uyt3br9h-ats.iot.us-east-2.amazonaws.com caching
[DEBUG] [2021-02-18T19:57:17Z] [b5754b40] [dns] - static: new address resolved 2600:1f00:6000::340f:777c for host a1ypu7uyt3br9h-ats.iot.us-east-2.amazonaws.com caching
[DEBUG] [2021-02-18T19:57:17Z] [b5754b40] [dns] - static: new address resolved 2600:1f00:6000::d3b:1e59 for host a1ypu7uyt3br9h-ats.iot.us-east-2.amazonaws.com caching
[DEBUG] [2021-02-18T19:57:17Z] [b684fb40] [task-scheduler] - id=0xb4e03050: Scheduling attempt_connection task for immediate execution
[DEBUG] [2021-02-18T19:57:17Z] [b684fb40] [task-scheduler] - id=0xb4e03b50: Scheduling attempt_connection task for immediate execution
[DEBUG] [2021-02-18T19:57:17Z] [b684fb40] [task-scheduler] - id=0xb4e03050: Running attempt_connection task with <Running> status
[DEBUG] [2021-02-18T19:57:17Z] [b684fb40] [socket] - id=0xb5d00b70 fd=7: initializing with domain 1 and type 0
[DEBUG] [2021-02-18T19:57:17Z] [b684fb40] [socket] - id=0xb5d00b70 fd=7: setting socket options to: keep-alive 0, keep idle 0, keep-alive interval 0, keep-alive probe count 0.
[WARN] [2021-02-18T19:57:17Z] [b684fb40] [socket] - id=0xb5d00b70 fd=7: setsockopt() for NO_SIGNAL failed with errno 92. If you are having SIGPIPE signals thrown, you may want to install a signal trap in your application layer.
[DEBUG] [2021-02-18T19:57:17Z] [b684fb40] [socket] - id=0xb5d00b70 fd=7: beginning connect.
[DEBUG] [2021-02-18T19:57:17Z] [b684fb40] [socket] - id=0xb5d00b70 fd=7: connecting to endpoint 2600:1f00:6000::383:1f34:443.
[ERROR] [2021-02-18T19:57:17Z] [b684fb40] [socket] - id=0xb5d00b70 fd=7: connect failed with error code 101.
[INFO] [2021-02-18T19:57:17Z] [b684fb40] [dns] - id=0xa42b310: recording failure for record 2600:1f00:6000::383:1f34 for a1ypu7uyt3br9h-ats.iot.us-east-2.amazonaws.com, moving to bad list
[DEBUG] [2021-02-18T19:57:17Z] [b684fb40] [dns] - static: purging address 2600:1f00:6000::383:1f34 for host a1ypu7uyt3br9h-ats.iot.us-east-2.amazonaws.com from the cache due to cache eviction or shutdown
[DEBUG] [2021-02-18T19:57:17Z] [b684fb40] [socket] - id=0xb5d00b70 fd=7: is still open, closing...
[DEBUG] [2021-02-18T19:57:17Z] [b684fb40] [socket] - id=0xb5d00b70 fd=7: closing
[ERROR] [2021-02-18T19:57:17Z] [b684fb40] [channel-bootstrap] - id=0xa44d200: failed to create socket with error 1049
[DEBUG] [2021-02-18T19:57:17Z] [b684fb40] [task-scheduler] - id=0xb4e03b50: Running attempt_connection task with <Running> status
[DEBUG] [2021-02-18T19:57:17Z] [b684fb40] [socket] - id=0xb5d00b70 fd=7: initializing with domain 0 and type 0
[DEBUG] [2021-02-18T19:57:17Z] [b684fb40] [socket] - id=0xb5d00b70 fd=7: setting socket options to: keep-alive 0, keep idle 0, keep-alive interval 0, keep-alive probe count 0.
[WARN] [2021-02-18T19:57:17Z] [b684fb40] [socket] - id=0xb5d00b70 fd=7: setsockopt() for NO_SIGNAL failed with errno 92. If you are having SIGPIPE signals thrown, you may want to install a signal trap in your application layer.
[DEBUG] [2021-02-18T19:57:17Z] [b684fb40] [socket] - id=0xb5d00b70 fd=7: beginning connect.
[DEBUG] [2021-02-18T19:57:17Z] [b684fb40] [socket] - id=0xb5d00b70 fd=7: connecting to endpoint 3.131.31.52:443.
[DEBUG] [2021-02-18T19:57:17Z] [b684fb40] [task-scheduler] - id=0xb5d01aa0: Scheduling (null) task for future execution at time 8129633861413
[DEBUG] [2021-02-18T19:57:17Z] [b684fb40] [task-scheduler] - id=0xb5d01ae0: Scheduling epoll_event_loop_unsubscribe_cleanup task for immediate execution
[INFO] [2021-02-18T19:57:17Z] [b684fb40] [socket] - id=0xb5d00b70 fd=7: connection success
[DEBUG] [2021-02-18T19:57:17Z] [b684fb40] [socket] - id=0xb5d00b70 fd=7: local endpoint 10.0.2.15:39292
[DEBUG] [2021-02-18T19:57:17Z] [b684fb40] [socket] - id=0xb5d00b70 fd=7: assigning to event loop 0xa4371f0
[DEBUG] [2021-02-18T19:57:17Z] [b684fb40] [channel-bootstrap] - id=0xa44d200: client connection on socket 0xb5d00b70 completed with error 0.
[DEBUG] [2021-02-18T19:57:17Z] [b684fb40] [channel] - id=0xb5d01c70: Beginning creation and setup of new channel.
[DEBUG] [2021-02-18T19:57:17Z] [b684fb40] [task-scheduler] - id=0xb5d01e30: Scheduling on_channel_setup_complete task for immediate execution
[DEBUG] [2021-02-18T19:57:17Z] [b684fb40] [task-scheduler] - id=0xb5d01ae0: Running epoll_event_loop_unsubscribe_cleanup task with <Running> status
[DEBUG] [2021-02-18T19:57:17Z] [b684fb40] [task-scheduler] - id=0xb5d01e30: Running on_channel_setup_complete task with <Running> status
[DEBUG] [2021-02-18T19:57:17Z] [b684fb40] [channel] - id=0xb5d01c70: setup complete, notifying caller.
[DEBUG] [2021-02-18T19:57:17Z] [b684fb40] [channel] - id=0xb5d01c70: no message pool is currently stored in the event-loop local storage, adding 0xb5d01f80 with max message size 16384, message count 4, with 4 small blocks of 128 bytes.
[DEBUG] [2021-02-18T19:57:17Z] [b684fb40] [channel-bootstrap] - id=0xa44d200: channel 0xb5d01c70 setup succeeded: bootstrapping.
[DEBUG] [2021-02-18T19:57:17Z] [b684fb40] [socket-handler] - id=0xb5d12620: Socket handler created with max_read_size of 16384
[DEBUG] [2021-02-18T19:57:17Z] [b684fb40] [task-scheduler] - id=0xb5d12718: Scheduling tls_timeout task for future execution at time 8134691976802
[DEBUG] [2021-02-18T19:57:17Z] [b684fb40] [tls-handler] - id=0xb5d12700: Alpn protocol negotiated as x-amzn-mqtt-ca
[DEBUG] [2021-02-18T19:57:17Z] [b684fb40] [tls-handler] - id=0xb5d12700: Remote server name is a1ypu7uyt3br9h-ats.iot.us-east-2.amazonaws.com
[DEBUG] [2021-02-18T19:57:17Z] [b684fb40] [channel-bootstrap] - id=0xa44d200: tls negotiation result 0 on channel 0xb5d01c70
[DEBUG] [2021-02-18T19:57:17Z] [b684fb40] [mqtt-client] - id=0xa450e40: Connection successfully opened, sending CONNECT packet
[DEBUG] [2021-02-18T19:57:17Z] [b684fb40] [task-scheduler] - id=0xb5d2c800: Scheduling mqtt_connack_timeout task for future execution at time 8127847022119
[DEBUG] [2021-02-18T19:57:17Z] [b684fb40] [mqtt-client] - id=0xa450e40: Adding username ?SDK=PythonV2&Version=0.9.15 to connection
Traceback (most recent call last):
  File "shadow.py", line 275, in <module>
[DEBUG] [2021-02-18T19:57:17Z] [b684fb40] [tls-handler] - id=0xb5d12700: Alert code 0
[DEBUG] [2021-02-18T19:57:17Z] [b684fb40] [task-scheduler] - id=0xb5d01d88: Scheduling channel_shutdown task for immediate execution
[DEBUG] [2021-02-18T19:57:17Z] [b684fb40] [task-scheduler] - id=0xb5d01d88: Running channel_shutdown task with <Running> status
[DEBUG] [2021-02-18T19:57:17Z] [b684fb40] [channel] - id=0xb5d01c70: beginning shutdown process
[DEBUG] [2021-02-18T19:57:17Z] [b684fb40] [channel] - id=0xb5d01c70: handler 0xb5d12620 shutdown in read dir completed.
[DEBUG] [2021-02-18T19:57:17Z] [b684fb40] [tls-handler] - id=0xb5d12700: Shutting down read direction with error code 0
[DEBUG] [2021-02-18T19:57:17Z] [b684fb40] [channel] - id=0xb5d01c70: handler 0xb5d12700 shutdown in read dir completed.
[DEBUG] [2021-02-18T19:57:17Z] [b684fb40] [channel] - id=0xb5d01c70: handler 0xa450e54 shutdown in read dir completed.
[DEBUG] [2021-02-18T19:57:17Z] [b684fb40] [task-scheduler] - id=0xb5d01c84: Scheduling (null) task for immediate execution
[DEBUG] [2021-02-18T19:57:17Z] [b684fb40] [task-scheduler] - id=0xb5d01c84: Running (null) task with <Running> status
[INFO] [2021-02-18T19:57:17Z] [b684fb40] [mqtt-client] - id=0xa450e40: sending disconnect message as part of graceful shutdown.
[DEBUG] [2021-02-18T19:57:17Z] [b684fb40] [mqtt-client] - id=0xa450e40: failed to send courteous disconnect io message
[DEBUG] [2021-02-18T19:57:17Z] [b684fb40] [channel] - id=0xb5d01c70: handler 0xa450e54 shutdown in write dir completed.
[DEBUG] [2021-02-18T19:57:17Z] [b684fb40] [tls-handler] - id=0xb5d12700: Shutting down write direction
[DEBUG] [2021-02-18T19:57:17Z] [b684fb40] [channel] - id=0xb5d01c70: handler 0xb5d12700 shutdown in write dir completed.
[DEBUG] [2021-02-18T19:57:17Z] [b684fb40] [socket] - id=0xb5d00b70 fd=7: closing
[DEBUG] [2021-02-18T19:57:17Z] [b684fb40] [task-scheduler] - id=0xb5d01c40: Scheduling epoll_event_loop_unsubscribe_cleanup task for immediate execution
[DEBUG] [2021-02-18T19:57:17Z] [b684fb40] [task-scheduler] - id=0xb5d12674: Scheduling socket_handler_close task for immediate execution
[DEBUG] [2021-02-18T19:57:17Z] [b684fb40] [task-scheduler] - id=0xb5d01c40: Running epoll_event_loop_unsubscribe_cleanup task with <Running> status
[DEBUG] [2021-02-18T19:57:17Z] [b684fb40] [task-scheduler] - id=0xb5d12674: Running socket_handler_close task with <Running> status
[DEBUG] [2021-02-18T19:57:17Z] [b684fb40] [channel] - id=0xb5d01c70: handler 0xb5d12620 shutdown in write dir completed.
[DEBUG] [2021-02-18T19:57:17Z] [b684fb40] [task-scheduler] - id=0xb5d01c84: Scheduling (null) task for immediate execution
[DEBUG] [2021-02-18T19:57:17Z] [b684fb40] [task-scheduler] - id=0xb5d01c84: Running (null) task with <Running> status
[DEBUG] [2021-02-18T19:57:17Z] [b684fb40] [channel] - id=0xb5d01c70: during shutdown, canceling task 0xb5d12718
[DEBUG] [2021-02-18T19:57:17Z] [b684fb40] [task-scheduler] - id=0xb5d12718: Running tls_timeout task with <Canceled> status
[DEBUG] [2021-02-18T19:57:17Z] [b684fb40] [channel] - id=0xb5d01c70: during shutdown, canceling task 0xb5d2c800
[DEBUG] [2021-02-18T19:57:17Z] [b684fb40] [task-scheduler] - id=0xb5d2c800: Running mqtt_connack_timeout task with <Canceled> status
[DEBUG] [2021-02-18T19:57:17Z] [b684fb40] [channel-bootstrap] - id=0xa44d200: channel 0xb5d01c70 shutdown with error 0.
[DEBUG] [2021-02-18T19:57:17Z] [b684fb40] [channel] - id=0xb5d01c70: destroying channel.
[DEBUG] [2021-02-18T19:57:17Z] [b684fb40] [channel-bootstrap] - id=0xa44d200: releasing bootstrap reference
    connected_future.result()
  File "/usr/lib/python3.7/concurrent/futures/_base.py", line 432, in result
    return self.__get_result()
  File "/usr/lib/python3.7/concurrent/futures/_base.py", line 384, in __get_result
    raise self._exception
awscrt.exceptions.AwsCrtError: AwsCrtError(name='AWS_ERROR_MQTT_UNEXPECTED_HANGUP', message='The connection was closed unexpectedly.', code=5134)
[DEBUG] [2021-02-18T19:57:17Z] [b7b6bec0] [mqtt-client] - id=0xa450e40: user called disconnect.
[ERROR] [2021-02-18T19:57:17Z] [b7b6bec0] [mqtt-client] - id=0xa450e40: Connection is not open, and may not be closed
[DEBUG] [2021-02-18T19:57:17Z] [b7b6bec0] [mqtt-client] - id=0xa450e40: Last refcount on connection has been released, start destroying the connection.
[DEBUG] [2021-02-18T19:57:17Z] [b7b6bec0] [mqtt-client] - id=0xa450e40: Destroying connection
[DEBUG] [2021-02-18T19:57:17Z] [b7b6bec0] [mqtt-topic-tree] - tree=0xa450f94: Cleaning up topic tree
[DEBUG] [2021-02-18T19:57:17Z] [b7b6bec0] [mqtt-client] - client=0xa4ee030: Cleaning up MQTT client
[DEBUG] [2021-02-18T19:57:17Z] [b7b6bec0] [channel-bootstrap] - id=0xa44d200: releasing bootstrap reference
[DEBUG] [2021-02-18T19:57:17Z] [b7b6bec0] [channel-bootstrap] - id=0xa44d200: releasing bootstrap reference
[DEBUG] [2021-02-18T19:57:17Z] [b7b6bec0] [channel-bootstrap] - id=0xa44d200: destroying

I feel so dumb right now, the error for me was that I had not activated the certificate in the IoT console. This is something stupid but could help others

@hackaprende thanks for sharing the solution to where you got stuck.

Posting here in case this helps any future searchers, I was also getting the error awscrt.exceptions.AwsCrtError: AWS_ERROR_MQTT_UNEXPECTED_HANGUP: The connection was closed unexpectedly. and for me, it turns out that I had not _activated_ the AWS IoT thing certificate. After creating it, there's another step to activate it.