Aws-iot-device-sdk-python-v2: configure offline publishing in v2

The current behavior is:

• All published messages are queued before sending, regardless of whether the client is offline or online.
• When the client gets online (or if it is already online) the messages from the queue are sent in order.
• HOWEVER: If the client WAS online, and goes offline, at that moment it cancels every PUBLISH and SUBSCRIBE in its queue.

We understand this is inconsistent. The user probably wants either:
1) messages are always queued until they can be sent
2) messages fail if the client is offline, or goes offline before they are sent.

We are very actively evaluating changing this. I will keep this issue open and we'll update when we've made changes (or formalized the current behavior)

If you want to take full control of your destiny right now, you could build a queue of your own that sends 1 message at a time to the client. If it succeeds, then send the next. If it fails, then retry. This also gives you the opportunity to have full control over the length of time a message can be queued, the number of messages that will be queued, etc.

If you want to take full control of your destiny right now, you could build a queue of your own that sends 1 message at a time to the client. If it succeeds, then send the next. If it fails, then retry. This also gives you the opportunity to have full control over the length of time a message can be queued, the number of messages that will be queued, etc.

Hi @graebm. How would you go about that? If my device publishes messages, but it's offline, I would think those messages stay in soms sort of queue? Is there a way to access this queue and purge certain messages from it if they have not been published yet?

Thank you

@samvandamme you can set quality of service (qos) to at most once (0).
qos=mqtt.QoS.AT_MOST_ONCE
On the same device subscribe to the topic that you are publishing to. Then to make sure messages are sent you maintain a list of messages that you have published. Only remove them when you have received back the message on the topic.

The current behavior is:

• All published messages are queued before sending, regardless of whether the client is offline or online.
• When the client gets online (or if it is already online) the messages from the queue are sent in order.
• HOWEVER: If the client WAS online, and goes offline, at that moment it cancels every PUBLISH and SUBSCRIBE in its queue.

We understand this is inconsistent. The user probably wants either:

1. messages are always queued until they can be sent
2. messages fail if the client is offline, or goes offline before they are sent.

We are very actively evaluating changing this. I will keep this issue open and we'll update when we've made changes (or formalized the current behavior)

If you want to take full control of your destiny right now, you could build a queue of your own that sends 1 message at a time to the client. If it succeeds, then send the next. If it fails, then retry. This also gives you the opportunity to have full control over the length of time a message can be queued, the number of messages that will be queued, etc.

So what's the recommended approach to handle devices going offline for some periods of time?

Assuming I have a device sending messages every 5 seconds, the device is online, messages are reaching AWS IoT Core. Then, the device lost connection for 10 minutes and then goes online again. What happens with all those messages that were tried to publish during the 10 minutes offline period? Do they get published as soon as the device goes online again or are they lost for ever?

The first few messages will be lost while the sdk hasn't realized that it is no longer disconnected. All subsequent messages would be save in the queue and sent when connection is restored.

When testing this I lost the first 5 messages when using the pubsub, which sends messages every second. So you would only loose one message most of the time.

I'd also like to understand this in greater detail. As mentioned above, messages that are published while offline go into the queue and then are published when the connection goes back online.

How many messages are stored in the offline queue? The previous SDK allowed you to choose, infinite, none, or a specified amount. I can't find any documentation about this. It sounds like infinite but I'd like to know. The previous SDK also allowed you to drop from the top or bottom of the queue. Is that possible now?

The previous SDK also allowed you to choose a drain rate for messages stored in the queue. Is this configurable now? If not, what is the value?