Lorawan-stack: Confirm end device session based on network server messages

I am in favor of this, as we have already discussed.
Application Server should always trust Network Server, because it always has the most up-to-date data about the device session.

• End Device sends FPort=0 uplink - AS won't receive this uplink

Shouldn't we change this so that NS does send this, but with empty payload and FPort 0, so that it won't be sent upstream?

• End Device sends FPort=0 uplink - AS won't receive this uplink

Shouldn't we change this so that NS does send this, but with empty payload and FPort 0, so that it won't be sent upstream?

It's redundant, since NS->AS messaging is async, queue invalidation can arrive before the FPort==0 uplink is sent to AS to confirm the session, so we have to do this anyway. The only reason to send an uplink to AS in response to FPort==0 uplink to NS, would be to ensure AS is notified of session change as soon as possible, but we don't have such need. Even then it would make way more sense to introduce a SessionSwitch message, which NS would send to AS instead of the FPort 0 uplink.

I have upgraded the priority to prio/high as it is affecting v3.11.1 deployments.

The changes should be the following:

• Add SessionKeyID to DownlinkQueueInvalidation. If the queue is empty, the AS cannot know at this moment which queue was invalidated.

The following proto addition (field 3) should suffice.

message ApplicationInvalidatedDownlinks {
  repeated ApplicationDownlink downlinks = 1;
  uint32 last_f_cnt_down = 2;
  bytes session_key_id = 3 [(gogoproto.customname) = "SessionKeyID", (validate.rules).bytes.max_len = 2048];
}

• The AS should update the queue + FCnt of the correct session, instead of always assuming that the invalidation is about the dev.Session.

https://github.com/TheThingsNetwork/lorawan-stack/blob/e2fa6c085eaaf1a0b70939020244875bd01e5857/pkg/applicationserver/applicationserver.go#L1020-L1038

Instead of using the dev.Session always, do a switch on SessionKeyID in order to establish which session to use. If required, update the current dev.Session.

• The AS should check which session a nacked message is from - it is possible that the nacked message is from a pending session (from the AS perspective) and as such the FCnt update should be done on the correct session.

https://github.com/TheThingsNetwork/lorawan-stack/blob/e2fa6c085eaaf1a0b70939020244875bd01e5857/pkg/applicationserver/applicationserver.go#L1060-L1073

As before, do a switch on the SessionKeyID in order to determine which session to use. If required, update the current dev.Session.

• (Optional) return error details on DownlinkQueue{Push|Replace} with the minimum FCnt and always update the LastAFCntDown to this value. This would ensure that the system converges if at any point we're for some reason out of sync between AS and NS.

The following proto addition should be filled and be added as error details to the errFCntTooLow in the NS:

message UpdateDownlinkQueueErrorDetails {
  bytes session_key_id = 1 [(gogoproto.customname) = "SessionKeyID", (validate.rules).bytes.max_len = 2048];
  uint32 last_f_cnt_down = 2;
}

The AS can then take these details and update the current session.

The changes are backwards compatible and hopefully minimal on the NS side. The genie is already out of the bottle - the whole protocol between AS and NS slowly became asynchronous, and simply reverting the FPort=0 change won't be enough. I don't think that this transformation was wrong at the end of the day, but we must fix these quirks regarding session bisimulation.

cc @johanstokking, @rvolosatovs

Sounds good to me!

Anything I can do here? If so, please re-assign me and let me know what.

Anything I can do here? If so, please re-assign me and let me know what.

I'm already working on this, with a huge emphasis on the following point:

* (Optional) return error details on `DownlinkQueue{Push|Replace}` with the minimum `FCnt` and always update the `LastAFCntDown` to this value. This would ensure that the system converges if at any point we're for some reason out of sync between AS and NS.

The reason for this is that taking actions based on the events received from the NS, as part of the queue, is fundamentally not really enough:

• Messages may be old, but we want to do fixes to the queue _now_
• Messages may be lost - the queue in not infinite
• Messages may be reordered - they are pushed in asynchronously into the queue, and they are popped in potentially reordered ways

Given these characteristics, there are two options:

• Delay the queue computation to the end of the batch processing (basically if you receive 10 downlink queue invalidations, you recalculate the queue only at the end). This is not enough due to the following reasons:
  
  
  
  • When the AS processes a batch of messages, it still doesn't know if he is _really_ at the end of the queue, or processing some batch in the middle
  
  
  • Messages may still be lost, and determining their order is not trivial.
  
  
• Just _trust the NS_. What I mean by this, is that push/replace operations return the dev.Session.SessionKeyID, dev.PendingSession.SessionKeyID, and the LastAFCntDown, as part of the error details. We then use the error details to rebuild the session in the AS. Fundamentally this means that when we try to do a downlink queue operation, using outdated data (perhaps an outdated session, perhaps a FCnt too low), we eventually converge to the NS state. It may take one, two, three tries, I'll make it bounded in order to avoid infinitely spinning, but we're at least operating with information that's significantly newer than the one from the uplink messages queue.

• Just _trust the NS_. What I mean by this, is that push/replace operations return the dev.Session.SessionKeyID, dev.PendingSession.SessionKeyID, and the LastAFCntDown, as part of the error details. We then use the error details to rebuild the session in the AS. Fundamentally this means that when we try to do a downlink queue operation, using outdated data (perhaps an outdated session, perhaps a FCnt too low), we eventually converge to the NS state. It may take one, two, three tries, I'll make it bounded in order to avoid infinitely spinning, but we're at least operating with information that's significantly newer than the one from the uplink messages queue.

I think this is the best option.