Product-apim: Setting up WSO2 API Manager behind the reverse proxy issue

Created on 12 Nov 2019  ·  39Comments  ·  Source: wso2/product-apim

In the configuration files I set, hostname and transport.https.properties in the deployment.toml file.
I have configured Nginx exactly like the below documents:
here
but it doesn't work. The problem is that as I try to open the publisher portal it asks for login and after the logging in, with correct username and password, it can not introspect this url receives HTTP-500 error, so it redirects to Login page and since I have logged in, it redirects me to API page and since it can not introspect, it redirects me to login page and this loop goes on forever!

I wanted to know what I'm missing and how it could be configurated to resolve this problem?

Affecte3.0.0 TypQuestion

All 39 comments

Here is a similar issue

Hi @amir-dh,
Could you please try enabling the reverse proxy in
wso2am3.0.0/repository/deployment/server/jaggeryapps/publisher/site/public/theme/defaultTheme.js

const REVERSE_PROXY = {
enabled: true,
origin: 'api.am.wso2.com',
};

@isharac
I did set reverse proxy

const REVERSE_PROXY = {
enabled: true,
origin: 'MyDomain',
};

but again doesn't work. when I open the publisher portal it gives error invalid_callback

@amir-dh,
You need to update the callback URLs accordingly.

Is this a stand alone deployment?
We have already identifies issue in NGINX ingress, but if it is a standalone deployment, it should by chaging the callback urls.
Please refer my blog [1] and change the callback URLs of admin_publisher and admin_store SP applications according to your hostname.

Sorry for the inconvenience but our docs are in WIP. We will update the docs soon.

[1] https://ishara-cooray.blogspot.com/2019/11/how-to-change-devportal-to-different.html

@isharac
I update the callback URLs but doesn't work
after it I run the Nginx and ApiM 3.0 in the localhost and only I have enable reverse proxy and update callback URLs. publisher portal with set domain is worked
but when deploy sample API and click the "view in devportal" or "sign-in" the page URL is localhost:9443.

@amir-dh
You need to configure the devportal url in deployment.toml
To do that enable below configuration in /repository/conf/deployment.toml and set the correct devportal url as the url.

You can enable by removing the #.

[apim.devportal]

url = "https://localhost:${mgt.transport.https.port}/devportal"

@isharac
I set configurated this lines:

[apim.devportal]
url = "https://MyDomain/devportal"

and set the reverse proxy:

const REVERSE_PROXY = {
enabled: true,
origin: 'MyDomain',
};

but the sign-in and log-out page URL is localhost. on publisher and devportal

@amir-dh
could you please attach screenshots of the above pages including url.
Also the callback url of devportal SP(admin_admin_store)?

@isharac

  1. I go to devportal page and click the sign-in:

click-the-sign-in

  1. this URL is sign-in page (publisher and devportal):

localhost-devportal

I did in sign-in page MyDomain replaced with "localhost:9443" and it is opened

but when I update callback URL admin_admin_store and go to the sign-in page it give error invalid_callback

I am having the same issue as amir-dh

is there a way to manually edit the api-manager.xml to set the correct URL, and avoid being overwritten once the server is started?

regards,

JP

@amir-dh
Thanks, for the screenshots.
Looks like you are using a port offset 2 as well.
Can you please share the

/repository/deployment/jaggeryapps/devportal/site/public/theme/settings.js
and the reverse proxy configurations?

Meanwhile, I tried by changing the hostname and fronted by an Nginx .
The steps I followed are documented in the blog [1].
Please let us know if that does not work for you.

@JUANPARBELAEZ
Can you please check the steps in the same blog[1] and let me know any feedback?

[1] https://ishara-cooray.blogspot.com/2019/11/how-to-change-default-hostname-of-wso2.html

@isharac
Hi,
I run the raw APIM and set the configuration from here step by step, but it doesn't work.
it error in publisher and store on the sign-in page is 500 (this page isn't working), error in carbon after entering username and password and click the sign-in button is 403 (forbidden).
I read wso2carbon.log, publisher and store error is connection refused and it is no error for carbon

@amir-dh
Please share the files below to analyse after removing any sensitive data.

  1. wso2carbon.log
    2.wso2am3.0.0/repository/deployment/server/jaggeryapps/publisher/site/public/theme/defaultTheme.js
  2. wso2am3.0.0/repository/deployment/server/jaggeryapps/devportal/site/public/theme/settings.js
  3. nginx.conf
  4. wso2am3.0.0/repository/deployment.toml

@isharac

wso2.zip
Here you are.

@amir-dh ,
Sorry for getting back to you late.
Provided configurations look fine.

However, we have revamped reverse proxy configuration due to an issue identified in[1] in the latest release.
Could youi please retry in the latest release WSO2AM-3.1.0-M2.

[1] https://github.com/wso2/product-apim/issues/6817

From the APIM-3.1.0-M 2 onwards, to configure a reverse proxy, you will need to do the below configuration changes.

  1. Change the hostname in the deployment.toml
    [server]
    hostname = "api.am.wso2.com"

  2. reverse proxy configuration
    server {
    listen 443 ssl;
    server_name api.am.wso2.com;
    proxy_set_header X-Forwarded-Port 443;
    ssl_certificate /usr/local/etc/nginx/ssl/nginx.crt;
    ssl_certificate_key /usr/local/etc/nginx/ssl/nginx.key;
    location / {
    proxy_set_header X-Forwarded-Host $host;
    proxy_set_header X-Forwarded-Server $host;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_pass https://api.am.wso2.com:9443;
    }
    }

  3. Add host mapping in /etc/hosts
    127.0.0.1 . api.am.wso2.com

  4. Change the callback URLs by login to the Management console if you are not starting the server for the first time.
    https://api.am.wso2.com:9443/carbon/

Go to service providers and find admin_admin_publisher
Callback URL in publisher
regexp=(https://api.am.wso2.com:9443/publisher/services/auth/callback/login|https://api.am.wso2.com:9443/publisher/services/auth/callback/logout)

@isharac
I download the APIM-3.1.0-M2, changed the hostname in deployment.toml file and changed nginx config to:
server {
listen 443 ssl;
server_name myDomain;
proxy_set_header X-Forwarded-Port 443;
ssl_certificate /usr/local/etc/nginx/ssl/nginx.crt;
ssl_certificate_key /usr/local/etc/nginx/ssl/nginx.key;
location / {
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Server $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass https://myDomain:9443;
}
}

but it doesn't work.
when called publisher it redirected to here . As you can see it redirects me to MyDomain with the port. not the reverse proxy URI, for example, it redirects me from https://mydomain.com/publisher to https://mydomain.com:9445/oauth2/...... If I manually remove the port followed by mydomain, it will work properly.
This problem is the same for the devportal when I want to sign in, either.

For Management Console after the login, it redirected from login page to this url and gives error 403(forbidden)

FYI: I checked the issues that you sent for me; and on features/apimgt/org.wso2.carbon.apimgt.publisher.feature/src/main/resources/publisher/site/public/theme/defaultTheme.js this:

reverseProxy: {
            auto: true,
            forwardedHeader: 'x-forwarded-for',
            loopbackHost: 'localhost',
            defaultHTTPSPort: 9443,
        },

is available in the issue [1] but it is not available in [2]

[1] (https://github.com/wso2/carbon-apimgt/pull/7842/files/88efcf71b4aa4b8bf5511db8fc91655ba7969aa8?utf8=%E2%9C%93&diff=split&w=1)

[2] (https://github.com/wso2/carbon-apimgt/pull/7858/files)

Hi @amir-dh

I am sorry u need to add the proxy port as well in the deployment.toml

Please add the below configuration i n the /repository/conf/deployment.toml
[transport.https.properties]
proxyPort = 443

@isharac
I added the proxy port in the deployment file

I called https://mydomain.com/ it redirected to publisher portal
but when I called https://mydomain.com/publisher it redirects to https://**myIP**/publisher and I get this message This site can’t be reached. in devportal and carbon URLs i got same error

@amir-dh
Please find my comments.

''I called https://mydomain.com/ it redirected to publisher portal''
This is the default case.

Redirect to IP looks a bit strange.
Could you please share the nginx.conf and the HAR file downloaded from the browser console for further analysis?

Yiou may refer below to find the download button in chrome.
Screen Shot 2019-11-28 at 11 04 14 AM

myconf.zip
here you are

@amir-dh
Thanks for sharing the configurations.

In your nginx.conf proxyPass is configured with the IP.
Can you please change it to the domain as below and recheck?

proxy_pass https://myDomain:9443;

Thanks.

Hi @amir-dh ,

I couldn't see any redirection to an IP address in the given HAR. It seems that the request has failed due to

invalid_callback
--
Registered callback does not match with the provided url.

So the issue is most probably, The callback URL given in the admin_apim_publisher

image

it should have the following URL in the regex

https://mydomain.com/publisher/services/auth/callback/login

@isharac @tmkasun
thank you for helping me.
solved my problem with your tips.
I setting below config:

  1. change the hostname to mydomain in deployment file
  2. set Nginx config:
server {
listen 443 ssl;
server_name myDomain;
proxy_set_header X-Forwarded-Port 443;
ssl_certificate /usr/local/etc/nginx/ssl/nginx.crt;
ssl_certificate_key /usr/local/etc/nginx/ssl/nginx.key;
location / {
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Server $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass https://myDomain:9443;
}
}
  1. add proxy port in the deployment file:
    [transport.https.properties]
    proxyPort = 443

  2. change devportal URL and gwdevportal URL in the deployment file from localhost to mydomain.com and remove the port

  3. change the callback url in console management and remove the port.

@amir-dh
Glad that you were able to resolve the issue. :)

@isharac
I did follow all configured
And I saw the “ TypeError: Cannot read property "forwardedHeader" from undefined .... ”
When I try to sigh-in in $HOST/devportal/apis
the page will be redirect to $HOST/devportal/services/configs

Do you have any suggestions ?
Please kindly give me a reply

Env

nginx version: nginx/1.10.3
WSO2 APIM 3.1.0 M3

TID: [-1234] [devportal] [2019-12-03 02:19:55,194] ERROR {org.jaggeryjs.jaggery.core.manager.WebAppManager} - org.mozilla.javascript.EcmaError: TypeError: Cannot read property "forwardedHeader" from undefined (/devportal/services/login/idp.jag#28) org.jaggeryjs.scriptengine.exceptions.ScriptException: org.mozilla.javascript.EcmaError: TypeError: Cannot read property "forwardedHeader" from undefined (/devportal/services/login/idp.jag#28)
at org.jaggeryjs.scriptengine.engine.RhinoEngine.execScript(RhinoEngine.java:587)
at org.jaggeryjs.scriptengine.engine.RhinoEngine.exec(RhinoEngine.java:289)
at org.jaggeryjs.jaggery.core.manager.WebAppManager.exec(WebAppManager.java:589)
at org.jaggeryjs.jaggery.core.manager.WebAppManager.execute(WebAppManager.java:508)
at org.jaggeryjs.jaggery.core.JaggeryServlet.doGet(JaggeryServlet.java:24)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:634)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:741)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.apache.catalina.core.ApplicationDispatcher.invoke(ApplicationDispatcher.java:712)
at org.apache.catalina.core.ApplicationDispatcher.processRequest(ApplicationDispatcher.java:459)
at org.apache.catalina.core.ApplicationDispatcher.doForward(ApplicationDispatcher.java:352)
at org.apache.catalina.core.ApplicationDispatcher.forward(ApplicationDispatcher.java:312)
at org.jaggeryjs.jaggery.core.JaggeryFilter.doFilter(JaggeryFilter.java:76)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.wso2.carbon.ui.filters.cache.ContentTypeBasedCachePreventionFilter.doFilter(ContentTypeBasedCachePreventionFilter.java:53)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.apache.catalina.filters.HttpHeaderSecurityFilter.doFilter(HttpHeaderSecurityFilter.java:126)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:202)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:490)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)
at org.wso2.carbon.identity.context.rewrite.valve.TenantContextRewriteValve.invoke(TenantContextRewriteValve.java:80)
at org.wso2.carbon.identity.authz.valve.AuthorizationValve.invoke(AuthorizationValve.java:100)
at org.wso2.carbon.identity.auth.valve.AuthenticationValve.invoke(AuthenticationValve.java:74)
at org.wso2.carbon.tomcat.ext.valves.CompositeValve.continueInvocation(CompositeValve.java:99)
at org.wso2.carbon.tomcat.ext.valves.TomcatValveContainer.invokeValves(TomcatValveContainer.java:49)
at org.wso2.carbon.tomcat.ext.valves.CompositeValve.invoke(CompositeValve.java:62)
at org.wso2.carbon.tomcat.ext.valves.CarbonStuckThreadDetectionValve.invoke(CarbonStuckThreadDetectionValve.java:146)
at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:678)
at org.wso2.carbon.tomcat.ext.valves.CarbonContextCreatorValve.invoke(CarbonContextCreatorValve.java:57)
at org.wso2.carbon.tomcat.ext.valves.RequestCorrelationIdValve.invoke(RequestCorrelationIdValve.java:116)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)
at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:408)
at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:853)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1587)
at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.mozilla.javascript.EcmaError: TypeError: Cannot read property "forwardedHeader" from undefined (/devportal/services/login/idp.jag#28)
at org.mozilla.javascript.ScriptRuntime.constructError(ScriptRuntime.java:3687)
at org.mozilla.javascript.ScriptRuntime.constructError(ScriptRuntime.java:3665)
at org.mozilla.javascript.ScriptRuntime.typeError(ScriptRuntime.java:3693)
at org.mozilla.javascript.ScriptRuntime.typeError2(ScriptRuntime.java:3712)
at org.mozilla.javascript.ScriptRuntime.undefReadError(ScriptRuntime.java:3725)
at org.mozilla.javascript.ScriptRuntime.getObjectProp(ScriptRuntime.java:1483)
at org.jaggeryjs.rhino.devportal.services.login.c0._c_script_0(/devportal/services/login/idp.jag:28)
at org.jaggeryjs.rhino.devportal.services.login.c0.call(/devportal/services/login/idp.jag)
at org.mozilla.javascript.ContextFactory.doTopCall(ContextFactory.java:394)
at org.mozilla.javascript.ScriptRuntime.doTopCall(ScriptRuntime.java:3091)
at org.jaggeryjs.rhino.devportal.services.login.c0.call(/devportal/services/login/idp.jag)
at org.jaggeryjs.rhino.devportal.services.login.c0.exec(/devportal/services/login/idp.jag)
at org.jaggeryjs.scriptengine.engine.RhinoEngine.execScript(RhinoEngine.java:583)
... 50 more

Hi @loeb-ift

What is the APIM version you are using?
Can you please share the devportal/site/public/theme/settings.js

Hi @isharac
as following
root@wso2am-313:/tmp/20191203# vi /opt/wso2am-3.1.0-m3/repository/deployment/server/jaggeryapps/devportal/site/public/theme/settings.js

const Settings = {
app: {
context: '/devportal',
reverseProxy: {
enabled: true,
origin: 'https://MYHOST',
},
},
grantTypes: {
authorization_code: 'Code',
implicit: 'Implicit',
refresh_token: 'Refresh Token',
password: 'Password',
'iwa:ntlm': 'IWA-NTLM',
client_credentials: 'Client Credentials',
'urn:ietf:params:oauth:grant-type:saml2-bearer': 'SAML2',
'urn:ietf:params:oauth:grant-type:jwt-bearer': 'JWT',
kerberos: 'Kerberos',
},
};

THX so much

@loeb-ift
Did you do any changes in this file?

The app json should as follows.

app: {
    context: '/devportal',
    customUrl: {
        enabled: false,
        forwardedHeader: 'X-Forwarded-For',
    },
    origin: {
        host: 'localhost',
    },
},

Please refer [1] for the default configuration file.
[1] https://github.com/wso2/carbon-apimgt/blob/v6.5.399/features/apimgt/org.wso2.carbon.apimgt.store.feature/src/main/resources/devportal/site/public/theme/settings.js

ref the doc from [https://ishara-cooray.blogspot.com/2019/11/how-to-change-default-hostname-of-wso2.html]

I'd tried to add the reverse proxy setting and edit the config /devportal/site/public/theme/settings.js
I saw I lost the ' forwardedHeader: 'X-Forwarded-For','
but it seen no works !

now the setting.js is :

`const Settings = {
app: {
context: '/devportal',
reverseProxy: {
enabled: true,
origin: 'https://$MYHOST',
forwardedHeader: 'X-Forwarded-For',
},
},
};

grantTypes: {
    authorization_code: 'Code',
    implicit: 'Implicit',
    refresh_token: 'Refresh Token',
    password: 'Password',
    'iwa:ntlm': 'IWA-NTLM',
    client_credentials: 'Client Credentials',
    'urn:ietf:params:oauth:grant-type:saml2-bearer': 'SAML2',
    'urn:ietf:params:oauth:grant-type:jwt-bearer': 'JWT',
    kerberos: 'Kerberos',
 },

};

Apologize if the blog is misleading but as I have noted down in the blog, it applies only for APIM 3.0.0
Because we have to change the reverse proxy configuration in APIM 3.1.0-m2 onwards.
I will update the blog with the latest configs.

Please follow the steps in comment[1] to configure the reverse proxy in APIM 3.1.0-m3

[1] https://github.com/wso2/product-apim/issues/6808#issuecomment-559584220

@isharac
appreciate your message
Actually,the blog did lots of help
but I miss the important point
A summary of installation

OS: Debian 4.9.189-3+deb9u2 (2019-11-11) x86_64 GNU/Linux
nginx version: nginx/1.10.3 with WSO2 3.1.0 M3
Install Step.
[https://github.com/wso2/product-apim/issues/6808#issuecomment-559584220]

It works now

Hello,
I have the same problem with nginx fronting (WSO2 APIM 3.0)
When i try to open publisher or devportal il have the follow error in my console
-Error: Error: No partial token found,redirecting to Anonymous View and User not found. Token refreshing failed.
/admin and /carbon work well

nginx version: nginx/1.16.1 in fronting server
WSO2 - APIM is isntaller an other server (CentOS Linux release 7.7.1908)
Isntalling WSO2 APIM 3.0 by rpm (fin in WSO2 site)

Network flow 9443,443 and 8243 is open and Nginx Fronted and WSO2 APIM 3.0 Server

Could you help me, i don't any solution :(

Thanks

Sam

Hi @SamWso2 ,

I think you are referring to these log messages,

image

These are some info logs printed out to identify the login state, You will see these logs in normal setup as well

Can you explain more about your issue ? or provide a HTTP Archive (HAR) file to analyze the issue ?

If you experienced a continues login loop scenario you can follow @isharac 's blog to configure the reverse proxy in APIM 3.0.0

https://ishara-cooray.blogspot.com/2019/11/how-to-change-devportal-to-different.html

Hi tmkasun,

In internal I can sign in /devportan and /publisher
In external, il use public URL (configured in NGINX) an i have access in Carbon and admin portal
I have access too in devportal URL but i can't sign in, its loop .....and i get nothing
I have WSO2 APIM 2.6 with NGINcXin my production environment and it work's very well.

Thanks

I use Squid proxy (and Nginx reverse proxy in front) to go to internet but i dont't implement this parameter in WSO2 configuration.
How should I do to configure proxy to go out internet on wSO2 APIM 3.0 ?

Thanks

I have put

but il have erros below

TID: [-1234] [devportal] [2020-03-10 10:45:20,910] ERROR {org.jaggeryjs.jaggery.core.manager.WebAppManager} - org.mozilla.javascript.WrappedException: Wrapped org.jaggeryjs.scriptengine.exceptions.ScriptException: java.net.ConnectException: Connection refused (Connection refused) (<>/META-INF/scripts/http.js#220) org.jaggeryjs.scriptengine.exceptions.ScriptException: org.mozilla.javascript.WrappedException: Wrapped org.jaggeryjs.scriptengine.exceptions.ScriptException: java.net.ConnectException: Connection refused (Connection refused) (<>/META-INF/scripts/http.js#220)

I use nginx server as revers-proxy in frontend = OK
But il use squid proxy to go out in internet

How should I do to configure proxy squid to go out internet on wSO2 APIM 3.0 configuration ? (no NGINX Rerverse proxy but squi proxy)

Thanks

hi @isharac
i am using wso2am 3.1.0 i want to rout the wso2 using nginx . I have configured the wso2 deployment.toml with hostname and pory port and configured the nginx but still i am facing the issue .can you please help me in this

Nginx is hosted in one VM and wso2 is avaialble in other VM

Was this page helpful?
0 / 5 - 0 ratings