Product-apim: In Memory Subscription Validation

@isharac

load information required for subscription validation to memory at server startup and at tenant loading for tenants.

In the PR wso2/carbon-apimgt#8563 the services are not secured at the moment. How these services will be secured? Basic auth or OAuth?

@isharac @chamilaadhi
Also with this change, exactly what attributes do we lose from below JWT and key validation response payloads.

{
  "aud": "http://org.wso2.apimgt/gateway",
  "sub": "[email protected]",
  "application": {
    "owner": "admin",
    "tierQuotaType": "requestCount",
    "tier": "10PerMin",
    "name": "New App",
    "id": 2,
    "uuid": null
  },
  "scope": "am_application_scope default",
  "iss": "https://localhost:9443/oauth2/token",
  "tierInfo": {
    "Unlimited": {
      "tierQuotaType": "requestCount",
      "stopOnQuotaReach": true,
      "spikeArrestLimit": 0,
      "spikeArrestUnit": null
    }
  },
  "keytype": "PRODUCTION",
  "subscribedAPIs": [
    {
      "subscriberTenantDomain": "carbon.super",
      "name": "PizzaShackAPI",
      "context": "/pizzashack/1.0.0",
      "publisher": "admin",
      "version": "1.0.0",
      "subscriptionTier": "Unlimited"
    }
  ],
  "consumerKey": "some_value",
  "exp": 3738643151,
  "iat": 1591159504,
  "jti": "some_valu2"
}

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
    <soapenv:Body>
        <ns:validateKeyResponse xmlns:ns="http://org.apache.axis2/xsd">
            <ns:return xmlns:ax2129="http://keymgt.apimgt.carbon.wso2.org/xsd"
                       xmlns:ax2131="http://api.apimgt.carbon.wso2.org/xsd"
                       xmlns:ax2133="http://dto.impl.apimgt.carbon.wso2.org/xsd"
                       xmlns:ax2135="http://model.api.apimgt.carbon.wso2.org/xsd"
                       xmlns:ax2136="http://dto.api.apimgt.carbon.wso2.org/xsd"
                       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="ax2133:APIKeyValidationInfoDTO">
                <ax2133:apiName>$APINAME</ax2133:apiName>
                <ax2133:apiPublisher>admin</ax2133:apiPublisher>
                <ax2133:apiTier xsi:nil="true" />
                <ax2133:applicationId>$APPLICATION_ID</ax2133:applicationId>
                <ax2133:applicationName>$APPLICATION_NAME</ax2133:applicationName>
                <ax2133:applicationTier>$APPLICATION_TIER</ax2133:applicationTier>
                <ax2133:authorized>true</ax2133:authorized>
                <ax2133:authorizedDomains xsi:nil="true" />
                <ax2133:consumerKey>fxzmLefepgKF2Qa</ax2133:consumerKey>
                <ax2133:contentAware>false</ax2133:contentAware>
                <ax2133:endUserName>[email protected]</ax2133:endUserName>
                <ax2133:endUserToken xsi:nil="true" />
                <ax2133:issuedTime>15288163</ax2133:issuedTime>
                <ax2133:scopes>default</ax2133:scopes>
                <ax2133:scopes>am_application_scope</ax2133:scopes>
                <ax2133:spikeArrestLimit>0</ax2133:spikeArrestLimit>
                <ax2133:spikeArrestUnit xsi:nil="true" />
                <ax2133:stopOnQuotaReach>true</ax2133:stopOnQuotaReach>
                <ax2133:subscriber>admin</ax2133:subscriber>
                <ax2133:subscriberTenantDomain>carbon.super</ax2133:subscriberTenantDomain>
                <ax2133:throttlingDataList>api_level_throttling_key</ax2133:throttlingDataList>
                <ax2133:tier>$TIER</ax2133:tier>
                <ax2133:type>$KEY_TYPE</ax2133:type>
                <ax2133:userType>APPLICATION</ax2133:userType>
                <ax2133:validationStatus>0</ax2133:validationStatus>
                <ax2133:validityPeriod>9223376854775807</ax2133:validityPeriod>
            </ns:return>
        </ns:validateKeyResponse>
    </soapenv:Body>
</soapenv:Envelope>

@isharac

load information required for subscription validation to memory at server startup and at tenant loading for tenants.

In the PR wso2/carbon-apimgt#8563 the services are not secured at the moment. How these services will be secured? Basic auth or OAuth?

This is secured with basic auth.

@praminda we will have only standard JWT claims in a JWT came from Oauth provider after this change

Ok. So we'll lose below attributes from JWT,

"application": {},
"tierInfo": {},
"keytype": "",
"subscribedAPIs": [],

and only following attributes will be available in key validation response. Am I correct?

<ax2133:apiName>$APINAME</ax2133:apiName>
<ax2133:apiPublisher>admin</ax2133:apiPublisher>
<ax2133:apiTier xsi:nil="true" />
<ax2133:authorized>true</ax2133:authorized>
<ax2133:authorizedDomains xsi:nil="true" />
<ax2133:consumerKey>fxzmLefepgKF2Qa</ax2133:consumerKey>
<ax2133:contentAware>false</ax2133:contentAware>
<ax2133:endUserName>[email protected]</ax2133:endUserName>
<ax2133:endUserToken xsi:nil="true" />
<ax2133:issuedTime>15288163</ax2133:issuedTime>
<ax2133:scopes>default</ax2133:scopes>
<ax2133:scopes>am_application_scope</ax2133:scopes>
<ax2133:userType>APPLICATION</ax2133:userType>
<ax2133:validationStatus>0</ax2133:validationStatus>
<ax2133:validityPeriod>9223376854775807</ax2133:validityPeriod>

This is implemented.

Any considerations to allow the user to configure this behaviour or not? Would much rather be able to get all info in a JWT than to rely on JMS-subscriptions to keep up-to-date. Or at least a way to generate a JWT with all the subscriptions from the database? (Specifically when used together with micro-gw).

Hi @christian-morin
We do not have plans to make it configurable as this is the default behavior now.
And third-party key manager doesn't aware of subscriptions metadata (as the DB is not shared), JWT generated from the third-party KM cannot be included the subscription data.

If you can share your specific requirement maybe we can come up with a solution.

The requirement for us is to use the Micro-Gateway alongside the APIM. The users generates the JWT in the APIM and use them to get access to the individual Micro-GW:s. Previously all the subscribedAPIs were present in the JWT-metadata and could be verified "offline" by the Micro-GW. This new setup requires all our Micro-GW-instances to listen to the JMS-subscription to keep up to date? (and no way to verify in the JWT which subscriptions are available to the client)

I'm just thinking that with potentially hundreds of micro-GW:s it'll be a hazzle to debug why someone is getting a 401...did the Topic not register, do we need to restart the Micro-GW to make it download all the subscription-metadata, what happens if the APIM isn't responding when the Micro-GW is starting etc. Having all the subscribedAPI:s and metadata attached in the JWT-metadata made the solution much more "robust" and the Micro-GW:s didn't have any ties with the APIM as the JWT held all the information (and the client could themselves check in the JWT-metadata which subscriptions they had access to with the key). So all I want is a way to generate the old behaviour JWT to use with our Micro-GW:s. Maybe it'll be possible by extending some JWT-Generator-class for APIM? (Or will this new handling make that very hard to accomplish?)

Hi @christian-morin
To support your usecase we can come up with 2 options.

Option 1:
Use APIM 3.1.0 with MG 3.2.0

Option 2:
Use API-M 3.2.0 and customize the TokenIssuer as explained in [1].
In this case, you will need to share the APIM db with the Key Manager node and will be able to add the required claims in the JWT.

Hope this helps!
[1] https://is.docs.wso2.com/en/latest/learn/extension-points-for-oauth/#oauth-token-generator

Hi @isharac

Thank you for the reply. I assume Option 1 isn't a very good long-term solution as we can then never upgrade the product? ;)

I suspected we would have to write a customized TokenIssuer as you suggest, but happy to know it should be possible. Thank you.