Apicurio-studio: API Edit Lost Connection on Kubernetes

Created on 7 Jun 2020  ·  29Comments  ·  Source: Apicurio/apicurio-studio

Hello, I am having similar difficulties to the issue explained in #813 when trying to run on Kubernetes exposing the WS via Ingress.

WebSocket connection to 'ws://apicurio-ws.192.168.1.5.nip.io/ws/designs/2?uuid=68064b1e-f839-4973-b65e-89660d4a07d9&user=admin&secret=eyJhbGciOiJSUzI1NiIsInR5cCIgOiAiSldUIiwia2lkIiA6ICJxRV9
COWQ4NFZS' failed: Error during WebSocket handshake: Unexpected response code: 404

I am using the kubernetes manifest files in the distro/kubernetes. I am only changing the host for every service.

picture cemnura

Most helpful comment

Great news! I assume we need a separate PR for these changes? What do you think @t-rap and @jsenko ?

picture EricWittmann on 23 Jun 2020
👍2

All 29 comments

I'm assuming the information in #813 didn't help you? This looks like the same problem. Maybe @0x218 could elaborate on the changes that worked for their installation? Or perhaps @t-rap has some thoughts? My kubernetes expertise is extremely limited.

EricWittmann picture EricWittmann on 7 Jun 2020

Yes I tried appyling the Version, Upgrade and Connection headers in the Nginx Ingress @0x218 suggested. Yet, I was not able to connect to the web socket.

Any logs I can provide to further understand the problem?

cemnura picture cemnura on 8 Jun 2020

If you can provide "full" browser console output and server logs from the -ws pod, that could help.

EricWittmann picture EricWittmann on 8 Jun 2020

browser log:

[ApiEditorPageComponent] Attempting to reconnect to the server.
main.a117148be57ecc117ff5.js:1 [ApisService] Getting an API Design: http://apicurio.192.168.1.5.nip.io/studio-api/designs/2
main.a117148be57ecc117ff5.js:1 [ApisService] Editing API Design: http://apicurio.192.168.1.5.nip.io/studio-api/designs/2/session
main.a117148be57ecc117ff5.js:1 [ApisService] Editing Session UUID: 0d50ffc8-1f69-41b0-b1f0-6c80a1771e00
main.a117148be57ecc117ff5.js:1 [ApisService] Content Version: 3
main.a117148be57ecc117ff5.js:1 [ApiEditorPageComponent] Definition loaded.  Opening editing session.
main.a117148be57ecc117ff5.js:1 [ApisService] Opening editing session on URL: ws://apicurio-ws.192.168.1.5.nip.io/ws/designs/2?uuid=0d50ffc8-1f69-41b0-b1f0-6c80a1771e00&user=admin&secret=eyJhbGciOiJSUzI1NiIsInR5cCIgOiAiSldUIiwia2lkIiA6ICJCNlRDTFAxdkIz
main.a117148be57ecc117ff5.js:1 WebSocket connection to 'ws://apicurio-ws.192.168.1.5.nip.io/ws/designs/2?uuid=0d50ffc8-1f69-41b0-b1f0-6c80a1771e00&user=admin&secret=eyJhbGciOiJSUzI1NiIsInR5cCIgOiAiSldUIiwia2lkIiA6ICJCNlRDTFAxdkIz' failed: Error during WebSocket handshake: Unexpected response code: 404
t.openEditingSession @ main.a117148be57ecc117ff5.js:1
(anonymous) @ main.a117148be57ecc117ff5.js:1
e.invoke @ polyfills.9c67d1be23abf75fea1a.js:1
onInvoke @ main.a117148be57ecc117ff5.js:1
e.invoke @ polyfills.9c67d1be23abf75fea1a.js:1
t.run @ polyfills.9c67d1be23abf75fea1a.js:1
(anonymous) @ polyfills.9c67d1be23abf75fea1a.js:1
e.invokeTask @ polyfills.9c67d1be23abf75fea1a.js:1
onInvokeTask @ main.a117148be57ecc117ff5.js:1
e.invokeTask @ polyfills.9c67d1be23abf75fea1a.js:1
t.runTask @ polyfills.9c67d1be23abf75fea1a.js:1
y @ polyfills.9c67d1be23abf75fea1a.js:1
t.invokeTask @ polyfills.9c67d1be23abf75fea1a.js:1
_ @ polyfills.9c67d1be23abf75fea1a.js:1
m @ polyfills.9c67d1be23abf75fea1a.js:1
main.a117148be57ecc117ff5.js:1 [ApiEditingSession] WS connection to server CLOSED: CloseEvent {isTrusted: true, wasClean: false, code: 1006, reason: "", type: "close", …}
main.a117148be57ecc117ff5.js:1 [ApiEditorPageComponent] **Notice** editing session DROPPED!  Reason code: 1006
main.a117148be57ecc117ff5.js:1 [ApiEditorPageComponent] Attempting to reconnect to the server.
main.a117148be57ecc117ff5.js:1 [ApisService] Getting an API Design: http://apicurio.192.168.1.5.nip.io/studio-api/designs/2
main.a117148be57ecc117ff5.js:1 [ApisService] Editing API Design: http://apicurio.192.168.1.5.nip.io/studio-api/designs/2/session
main.a117148be57ecc117ff5.js:1 [ApisService] Editing Session UUID: 5eecf1e6-2a83-4940-bdb0-110240374093
main.a117148be57ecc117ff5.js:1 [ApisService] Content Version: 3
main.a117148be57ecc117ff5.js:1 [ApiEditorPageComponent] Definition loaded.  Opening editing session.
main.a117148be57ecc117ff5.js:1 [ApisService] Opening editing session on URL: ws://apicurio-ws.192.168.1.5.nip.io/ws/designs/2?uuid=5eecf1e6-2a83-4940-bdb0-110240374093&user=admin&secret=eyJhbGciOiJSUzI1NiIsInR5cCIgOiAiSldUIiwia2lkIiA6ICJCNlRDTFAxdkIz
main.a117148be57ecc117ff5.js:1 WebSocket connection to 'ws://apicurio-ws.192.168.1.5.nip.io/ws/designs/2?uuid=5eecf1e6-2a83-4940-bdb0-110240374093&user=admin&secret=eyJhbGciOiJSUzI1NiIsInR5cCIgOiAiSldUIiwia2lkIiA6ICJCNlRDTFAxdkIz' failed: Error during WebSocket handshake: Unexpected response code: 404
t.openEditingSession @ main.a117148be57ecc117ff5.js:1
(anonymous) @ main.a117148be57ecc117ff5.js:1
e.invoke @ polyfills.9c67d1be23abf75fea1a.js:1
onInvoke @ main.a117148be57ecc117ff5.js:1
e.invoke @ polyfills.9c67d1be23abf75fea1a.js:1
t.run @ polyfills.9c67d1be23abf75fea1a.js:1
(anonymous) @ polyfills.9c67d1be23abf75fea1a.js:1
e.invokeTask @ polyfills.9c67d1be23abf75fea1a.js:1
onInvokeTask @ main.a117148be57ecc117ff5.js:1
e.invokeTask @ polyfills.9c67d1be23abf75fea1a.js:1
t.runTask @ polyfills.9c67d1be23abf75fea1a.js:1
y @ polyfills.9c67d1be23abf75fea1a.js:1
t.invokeTask @ polyfills.9c67d1be23abf75fea1a.js:1
_ @ polyfills.9c67d1be23abf75fea1a.js:1
m @ polyfills.9c67d1be23abf75fea1a.js:1
main.a117148be57ecc117ff5.js:1 [ApiEditingSession] WS connection to server CLOSED: CloseEvent {isTrusted: true, wasClean: false, code: 1006, reason: "", type: "close", …}
main.a117148be57ecc117ff5.js:1 [ApiEditorPageComponent] **Notice** editing session DROPPED!  Reason code: 1006

ws pod full log:

06-10 14:46:08,780 DEBUG [io.undertow.request.security] (default task-1) Authentication result was ATTEMPTED for /ws/designs/2
2020-06-10 14:46:08,781 DEBUG [io.undertow.request] (default task-1) Matched default handler path /ws/designs/2
2020-06-10 14:46:11,291 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread Status <== SCANNING
2020-06-10 14:46:11,291 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread scanning
2020-06-10 14:46:11,291 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) Periodic recovery first pass at Wed, 10 Jun 2020 14:46:11
2020-06-10 14:46:11,291 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) processing /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction transactions
2020-06-10 14:46:11,291 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) Recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.CommitMarkableResourceRecordRecoveryModule@7e04aa62' first pass processed
2020-06-10 14:46:11,291 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) AtomicActionRecoveryModule first pass
2020-06-10 14:46:11,291 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) processing /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction transactions
2020-06-10 14:46:11,291 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) Recovery module 'com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule@3b062bc6' first pass processed
2020-06-10 14:46:11,291 DEBUG [com.arjuna.ats.txoj] (Periodic Recovery) TORecoveryModule - first pass
2020-06-10 14:46:11,291 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) Recovery module 'com.arjuna.ats.internal.txoj.recovery.TORecoveryModule@29e089e5' first pass processed
2020-06-10 14:46:11,291 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) Recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.SubordinateAtomicActionRecoveryModule@5f676b11' first pass processed
2020-06-10 14:46:11,291 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change IDLE->FIRST_PASS

2020-06-10 14:46:11,291 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule - first pass
2020-06-10 14:46:11,291 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change FIRST_PASS->BETWEEN_PASSES

2020-06-10 14:46:11,291 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) Recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.XARecoveryModule@5ad9faef' first pass processed
2020-06-10 14:46:20,674 DEBUG [io.undertow.request] (default I/O-2) Matched default handler path /ws/designs/2
2020-06-10 14:46:20,675 DEBUG [io.undertow.request.security] (default task-1) Attempting to authenticate /ws/designs/2, authentication required: false
2020-06-10 14:46:20,675 DEBUG [io.undertow.request.security] (default task-1) Authentication outcome was NOT_ATTEMPTED with method io.undertow.security.impl.CachedAuthenticatedSessionMechanism@59a16760 for /ws/designs/2
2020-06-10 14:46:20,675 DEBUG [io.undertow.request.security] (default task-1) Authentication result was ATTEMPTED for /ws/designs/2
2020-06-10 14:46:20,675 DEBUG [io.undertow.request] (default task-1) Matched default handler path /ws/designs/2
2020-06-10 14:46:21,291 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) Periodic recovery second pass at Wed, 10 Jun 2020 14:46:21
2020-06-10 14:46:21,291 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) Recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.CommitMarkableResourceRecordRecoveryModule@7e04aa62' second pass processed
2020-06-10 14:46:21,291 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) AtomicActionRecoveryModule second pass
2020-06-10 14:46:21,291 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) Recovery module 'com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule@3b062bc6' second pass processed
2020-06-10 14:46:21,291 DEBUG [com.arjuna.ats.txoj] (Periodic Recovery) TORecoveryModule - second pass
2020-06-10 14:46:21,291 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) Recovery module 'com.arjuna.ats.internal.txoj.recovery.TORecoveryModule@29e089e5' second pass processed
2020-06-10 14:46:21,292 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) Recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.SubordinateAtomicActionRecoveryModule@5f676b11' second pass processed
2020-06-10 14:46:21,292 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change BETWEEN_PASSES->SECOND_PASS

2020-06-10 14:46:21,292 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule - second pass
2020-06-10 14:46:21,292 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule.transactionInitiatedRecovery completed
2020-06-10 14:46:21,292 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule.resourceInitiatedRecovery completed
2020-06-10 14:46:21,292 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change SECOND_PASS->IDLE

2020-06-10 14:46:21,292 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) Recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.XARecoveryModule@5ad9faef' second pass processed
2020-06-10 14:46:21,292 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread Status <== INACTIVE
2020-06-10 14:46:21,292 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread backing off
cemnura picture cemnura on 10 Jun 2020

Man I have no idea what might be going wrong, then. I was going to ask you to make sure that the -ws and -api pods are connected to the same database. If not you would get an error connecting, but in that case the -ws pod log would show the error, and you wouldn't get a 404 I don't think.

It seems like it must be some sort of Ingress problem, and I don't know enough about k8s to help! :(

EricWittmann picture EricWittmann on 10 Jun 2020

I also tried to directly expose the ws pod directly via a NodePort. Yet I still had no success.

I am going to try again. Would increasing the log level help?

cemnura picture cemnura on 10 Jun 2020

It couldn't hurt. There was something in the -ws log about authentication that I didn't quite understand, since the -ws component doesn't have any auth enabled. Maybe increasing the log level would illuminate that more.

EricWittmann picture EricWittmann on 10 Jun 2020

What level would you think I should increase the log level to?

cemnura picture cemnura on 10 Jun 2020

Probably start with DEBUG and see what happens. :)

EricWittmann picture EricWittmann on 10 Jun 2020

I increased the log levels to debug. Yet, there is no beneficial log.

ws

cloak.adapters.OAuthRequestAuthenticator] (default task-2) there was no code
2020-06-13 22:20:10,628 DEBUG [org.keycloak.adapters.OAuthRequestAuthenticator] (default task-2) redirecting to auth server
2020-06-13 22:20:10,628 DEBUG [org.keycloak.adapters.undertow.ServletSessionTokenStore] (default task-3) session was null, returning null
2020-06-13 22:20:10,628 DEBUG [org.keycloak.adapters.OAuthRequestAuthenticator] (default task-3) there was no code
2020-06-13 22:20:10,628 DEBUG [org.keycloak.adapters.OAuthRequestAuthenticator] (default task-3) redirecting to auth server
2020-06-13 22:20:10,628 DEBUG [org.keycloak.adapters.OAuthRequestAuthenticator] (default task-2) callback uri: http://apicurio-studio-ui.192.168.1.5.nip.io/version.js
2020-06-13 22:20:10,628 DEBUG [org.keycloak.adapters.OAuthRequestAuthenticator] (default task-3) callback uri: http://apicurio-studio-ui.192.168.1.5.nip.io/config.js
2020-06-13 22:20:10,628 DEBUG [io.undertow.request.security] (default task-2) Authentication outcome was NOT_ATTEMPTED with method org.keycloak.adapters.wildfly.WildflyAuthenticationMechanism@26c8577b for /version.js
2020-06-13 22:20:10,628 DEBUG [io.undertow.request.security] (default task-2) Authentication result was ATTEMPTED for /version.js
2020-06-13 22:20:10,628 DEBUG [io.undertow.request.security] (default task-3) Authentication outcome was NOT_ATTEMPTED with method org.keycloak.adapters.wildfly.WildflyAuthenticationMechanism@26c8577b for /config.js
2020-06-13 22:20:10,628 DEBUG [io.undertow.request.security] (default task-3) Sending authentication challenge for HttpServerExchange{ GET /config.js}
2020-06-13 22:20:10,628 DEBUG [io.undertow.request] (default task-2) Matched default handler path /version.js
2020-06-13 22:20:10,628 DEBUG [org.keycloak.adapters.AuthenticatedActionsHandler] (default task-2) AuthenticatedActionsValve.invoke http://apicurio-studio-ui.192.168.1.5.nip.io/version.js
2020-06-13 22:20:10,629 DEBUG [org.keycloak.adapters.AuthenticatedActionsHandler] (default task-2) Policy enforcement is disabled.
2020-06-13 22:20:10,629 DEBUG [io.undertow.session] (default task-3) Created session with id oIOfXf1RPvC2PBpI4dTuVZKYk7f8BDg0XUAFHWrC for exchange HttpServerExchange{ GET /config.js}
2020-06-13 22:20:10,629 DEBUG [org.keycloak.adapters.OAuthRequestAuthenticator] (default task-3) Sending redirect to login page: http://keycloak-microcks.192.168.1.5.nip.io/auth/realms/Apicurio/protocol/openid-connect/auth?response_type=code&client_id=apicurio-studio&redirect_uri=http%3A%2F%2Fapicurio-studio-ui.192.168.1.5.nip.io%2Fconfig.js&state=3e4e092e-bc54-4802-a517-d3462de6f343&login=true&scope=openid
2020-06-13 22:20:10,629 DEBUG [io.undertow.request.security] (default task-3) Authentication result was CHALLENGE_SENT for /config.js
2020-06-13 22:21:00,918 DEBUG [io.undertow.request] (default I/O-2) Timing out idle connection from /10.1.94.1:41180
2020-06-13 22:21:00,918 DEBUG [io.undertow.request] (default I/O-2) Timing out idle connection from /10.1.94.1:41178
2020-06-13 22:21:48,749 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread Status <== SCANNING
2020-06-13 22:21:48,749 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread scanning
2020-06-13 22:21:48,749 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) Periodic recovery first pass at Sat, 13 Jun 2020 22:21:48
2020-06-13 22:21:48,750 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) processing /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction transactions
2020-06-13 22:21:48,750 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) Recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.CommitMarkableResourceRecordRecoveryModule@3bc2db45' first pass processed
2020-06-13 22:21:48,750 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) AtomicActionRecoveryModule first pass
2020-06-13 22:21:48,750 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) processing /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction transactions
2020-06-13 22:21:48,750 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) Recovery module 'com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule@7ec04259' first pass processed
2020-06-13 22:21:48,750 DEBUG [com.arjuna.ats.txoj] (Periodic Recovery) TORecoveryModule - first pass
2020-06-13 22:21:48,750 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) Recovery module 'com.arjuna.ats.internal.txoj.recovery.TORecoveryModule@1a9ffeac' first pass processed
2020-06-13 22:21:48,751 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) Recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.SubordinateAtomicActionRecoveryModule@1aa1c748' first pass processed
2020-06-13 22:21:48,751 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change IDLE->FIRST_PASS

2020-06-13 22:21:48,751 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule - first pass
2020-06-13 22:21:48,751 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change FIRST_PASS->BETWEEN_PASSES

2020-06-13 22:21:48,751 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) Recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.XARecoveryModule@5b07f3c7' first pass processed

I however noticed a browser log that may explain the problem.

main.a117148….js:1 [ApiEditingSession] WS connection to server CLOSED: 
CloseEvent {isTrusted: true, wasClean: false, code: 1006, reason: "", type: "close", …}
bubbles: false
cancelBubble: false
cancelable: false
code: 1006
composed: false
currentTarget: WebSocket {__zone_symbol__openfalse: Array(1), __zone_symbol__messagefalse: Array(1), __zone_symbol__ON_PROPERTYopen: ƒ, __zone_symbol__ON_PROPERTYmessage: ƒ, __zone_symbol__ON_PROPERTYclose: ƒ, …}
defaultPrevented: false
eventPhase: 0
isTrusted: true
path: []
reason: ""
returnValue: true
srcElement: WebSocket {__zone_symbol__openfalse: Array(1), __zone_symbol__messagefalse: Array(1), __zone_symbol__ON_PROPERTYopen: ƒ, __zone_symbol__ON_PROPERTYmessage: ƒ, __zone_symbol__ON_PROPERTYclose: ƒ, …}
target: WebSocket {__zone_symbol__openfalse: Array(1), __zone_symbol__messagefalse: Array(1), __zone_symbol__ON_PROPERTYopen: ƒ, __zone_symbol__ON_PROPERTYmessage: ƒ, __zone_symbol__ON_PROPERTYclose: ƒ, …}
timeStamp: 1177278.1400000094
type: "close"
wasClean: false
__proto__: CloseEvent

The web socket season is being closed with the error code 1006

cemnura picture cemnura on 14 Jun 2020

The server log you included is actually the -ui pod's log - not the -ws pod's log.

EricWittmann picture EricWittmann on 15 Jun 2020

The error code of 1006 means the web socket closed abnormally by the browser. :( And sadly the browser will most likely not give you a reason. Some info here:

https://stackoverflow.com/questions/19304157/getting-the-reason-why-websockets-closed-with-close-code-1006

What you could perhaps try is finding a websocket testing tool of some kind and trying to use that. It might give you some diagnostic information we can use.

EricWittmann picture EricWittmann on 15 Jun 2020

The logs where retrieved with the following command

kubectl logs apicurio-studio-ws-7c458cf8d8-cgscc

is there a specific log that I can search for?

cemnura picture cemnura on 15 Jun 2020

Do you have any recommendations for testing the web socket?

cemnura picture cemnura on 15 Jun 2020

I found a way to test out the web socket via https://websocket.org/echo.html.

I copied the ws connection code to my computer. What message should I send?

cemnura picture cemnura on 15 Jun 2020

That's really strange because the output you posted above clearly shows a keycloak auth redirect and other keycloak related output, as well as requests for config.js and version.js - those are all things that the UI does. The websocket component doesn't have Keycloak included at all. Could you perhaps fetch the log output from all three pods? That would be interesting. Perhaps there is actually a problem with the pod!

As for testing the web socket. Just try connecting and see what happens. If it's working you should get something like this:

image

If yours is broken at the "connect" stage then that initial connect will probably fail.

EricWittmann picture EricWittmann on 15 Jun 2020

I tested out the websocket by copying the code provided on websocket.org and saved it as websocket.html on my hard drive to test the ws pod on my local network. Unfortunately I was not able to connect to the ws in my local network. I was however able to connect the public websocket
wss://studio-ws.apicur.io/designs. Therefore, I assume the websocket test is viable.

I provided all the logs in the attachment

kubectl logs apicurio-studio-ui-5c6f5df485-lcdxp > apicurio-studio-ui.log
kubectl logs apicurio-studio-ws-77dc7f7b87-57dcs > apicurio-studio-ws.log
kubectl logs apicurio-studio-api-79d9d799cb-mgsmx > apicurio-studio-api.log

apicurio-studio-api.log
apicurio-studio-ui.log
apicurio-studio-ws.log

cemnura picture cemnura on 16 Jun 2020

I finally got it to work! I directly exposed the ws pod via NodePort. I am now certain that the problem is with the ingress configurations.

cemnura picture cemnura on 16 Jun 2020
👍1

Awesome!! Thanks for pushing through on this (not giving up). And sorry I couldn't be of more help. My kubernetes knowledge level is low. Can you post whatever YAML config you needed to add to get it working? We will eventually be creating an operator for Apicurio Studio, but I don't know exactly when that might happen.

EricWittmann picture EricWittmann on 16 Jun 2020

Well to be honest I did not make a specific yaml configuration.

I create a NodePort service which means that the ws pods 8080 port is exposed on the machine IP on a random port.

This is how I created a NodePort
kubectl expose deployment apicurio-studio-ws --type NodePort --name test

which gave me a random port of 32204 that can be accessed via the machine IP:32204.

kubectl get svc 

NAME                           TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)          AGE
test                           NodePort    10.152.183.2     <none>        8080:32204/TCP   10h

I still think the ingress nginx should be the preferred way to expose the websocket because the NodePort port is random and can not be pre configured in the apicurio-configmap.yaml.

Which in my case is

apicurio-configmap.yaml

apiVersion: v1
kind: ConfigMap
metadata:
  name: apicurio-configmap
data:
  apicurio-db-connection-url: jdbc:mysql://apicuriodb:3306/apicuriodb
  apicurio-kc-client-id: apicurio-studio
  apicurio-kc-realm: Apicurio
  apicurio-microcks-api-url: http://microcks-microcks.192.168.1.5.nip.io/api
  apicurio-microcks-client-id: microcks-serviceaccount
  apicurio-ui-editing-url: ws://192.168.1.5:32204/
  apicurio-ui-feature-microcks: "true"
  apicurio-ui-feature-share-with-everyone: "true"
  apicurio-ui-hub-api-url: http://apicurio.192.168.1.5.nip.io/studio-api
  apicurio-ui-logout-redirect-uri: /
  keycloak-url: http://keycloak-microcks.192.168.1.5.nip.io/auth
cemnura picture cemnura on 16 Jun 2020

OK very interesting. No idea why the nginx approach wasn't working for you. :( But glad you got something working!

EricWittmann picture EricWittmann on 17 Jun 2020

I found a way to expose the ws pod via ingress. This solution is much more configurable.

Screen Shot 2020-06-19 at 1 40 21 AM

In this solution I created a separate URL for the WS pod via ingress.

I'l provide a PR for evaluation.

cemnura picture cemnura on 19 Jun 2020
👍1

hey, sorry for being late to the party.

@cemnura what ingress are you using, and in which version?

There has been a lot of fixes and changes to the nginx-ingress-controller, if you use that.

one thing which is being weird about it, is websocket behavior ,especially when you try to switch between http/https.

@EricWittmann i can test if a patched version works (nginx-ingress-version updated to the latest one) if you'd like

t-rap picture t-rap on 21 Jun 2020

Any help you can provide would be appreciated, @t-rap

EricWittmann picture EricWittmann on 22 Jun 2020

Helllo @t-rap I am running a microk8s Kubernetes which uses the nginx-ingress-controller-amd64 as far as I understand.

Name:         nginx-ingress-microk8s-controller-nd8vb
Namespace:    ingress
Priority:     0
Node:         mccloud/192.168.1.5
Start Time:   Tue, 02 Jun 2020 23:12:28 +0300
Labels:       controller-revision-hash=59cb5dd586
              name=nginx-ingress-microk8s
              pod-template-generation=1
Annotations:  <none>
Status:       Running
IP:           192.168.1.5
IPs:
  IP:           192.168.1.5
Controlled By:  DaemonSet/nginx-ingress-microk8s-controller
Containers:
  nginx-ingress-microk8s:
    Container ID:  containerd://3219d168e8fbb190acd214ab651f781f6adf51adcc2302891636a5ff250ef15f
    Image:         quay.io/kubernetes-ingress-controller/nginx-ingress-controller-amd64:0.25.1
    Image ID:      sha256:2b8ed1f2046d4b37c18cca2ecc4f435b6618d2d198c0c8bf617954e863cc5832
    Ports:         80/TCP, 443/TCP
    Host Ports:    80/TCP, 443/TCP
    Args:
      /nginx-ingress-controller
      --configmap=$(POD_NAMESPACE)/nginx-load-balancer-microk8s-conf
      --publish-status-address=127.0.0.1
    State:          Running
      Started:      Fri, 12 Jun 2020 15:08:07 +0300
    Last State:     Terminated
      Reason:       Completed
      Exit Code:    0
      Started:      Wed, 10 Jun 2020 23:37:02 +0300
      Finished:     Fri, 12 Jun 2020 15:08:03 +0300
    Ready:          True
    Restart Count:  187
    Liveness:       http-get http://:10254/healthz delay=30s timeout=5s period=10s #success=1 #failure=3
    Environment:
      POD_NAME:       nginx-ingress-microk8s-controller-nd8vb (v1:metadata.name)
      POD_NAMESPACE:  ingress (v1:metadata.namespace)
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from nginx-ingress-microk8s-serviceaccount-token-r6md6 (ro)
Conditions:
  Type              Status
  Initialized       True 
  Ready             True 
  ContainersReady   True 
  PodScheduled      True 
Volumes:
  nginx-ingress-microk8s-serviceaccount-token-r6md6:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  nginx-ingress-microk8s-serviceaccount-token-r6md6
    Optional:    false
QoS Class:       BestEffort
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/disk-pressure:NoSchedule
                 node.kubernetes.io/memory-pressure:NoSchedule
                 node.kubernetes.io/network-unavailable:NoSchedule
                 node.kubernetes.io/not-ready:NoExecute
                 node.kubernetes.io/pid-pressure:NoSchedule
                 node.kubernetes.io/unreachable:NoExecute
                 node.kubernetes.io/unschedulable:NoSchedule
Events:          <none>
cemnura picture cemnura on 22 Jun 2020

thank you very much @cemnura
could you test the fork i made (see beneath)? Thanks (again) in advance :)

i tested the configuration on https://github.com/t-rap/apicurio-studio-981/ with an ingress-nginx-controller version 0.24.1 and apicurio-studio 0.2.44.Final
i added the subPath "/designs" to ingress apicurio-studio-ws as this is a subpath which needs to be handled by ws.

I'll run some tests on ingress-controller 0.25.1 and apicurio-studio 0.2.46.Final probably on the weekend.
As the ingress-controller is maintained by kubernetes there's a load of changes to recognize.

i also deleted the configuration-snippet as it is not necessary anymore (my bad, duh)

I'll give an update when i'm finished.

t-rap picture t-rap on 22 Jun 2020
👍2

I will give it a try @t-rap 👍

cemnura picture cemnura on 22 Jun 2020
👍2

I was able to expose the ws pod on the same url with the ui pod using the Kubernetes manifest @t-rap provided.

However, with a few minor changes.

I changed the ws pod ingress name to apicurio-studio-ws

https://github.com/Apicurio/apicurio-studio/blob/17a8e6bc21cae5a90aebb1a7a0e64350aeb30f57/distro/kubernetes/apicurio-studio-ingresses.yaml#L38-L41

deleted the /ws suffix in the apicurio-configmap.yaml for apicurio-ui-editing-url key

https://github.com/Apicurio/apicurio-studio/blob/17a8e6bc21cae5a90aebb1a7a0e64350aeb30f57/distro/kubernetes/apicurio-configmap.yaml#L9

deleted the /ws path in the ingress configuration in the apicurio-studio-ingresses.yaml

Here is my configurations

apicurio-configmap.yaml

apiVersion: v1
kind: ConfigMap
metadata:
  name: apicurio-configmap
data:
  apicurio-db-connection-url: jdbc:mysql://apicuriodb:3306/apicuriodb
  apicurio-kc-client-id: apicurio-studio
  apicurio-kc-realm: Apicurio
  apicurio-microcks-api-url: http://microcks-microcks.192.168.1.5.nip.io/api
  apicurio-microcks-client-id: microcks-serviceaccount
  apicurio-ui-editing-url: ws://apicurio.192.168.1.5.nip.io
  apicurio-ui-feature-microcks: "true"
  apicurio-ui-feature-share-with-everyone: "true"
  apicurio-ui-hub-api-url: http://apicurio.192.168.1.5.nip.io/studio-api
  apicurio-ui-logout-redirect-uri: /
  keycloak-url: http://keycloak-microcks.192.168.1.5.nip.io/auth

apicurio-studio-ingresses.yaml

kind: Ingress
apiVersion: extensions/v1beta1
metadata:
  name: apicurio-studio-api
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /$1
  labels:
    module: apicurio-studio-api
spec:
  rules:
  - host: apicurio.192.168.1.5.nip.io
    http:
      paths:
      - backend:
          serviceName: "apicurio-studio-api"
          servicePort: 8091
        path: /studio-api/?(.*)
---
kind: Ingress
apiVersion: extensions/v1beta1
metadata:
  name: apicurio-studio-ui
  labels:
    module: apicurio-studio-ui
spec:
  rules:
    - host: apicurio.192.168.1.5.nip.io
      http:
        paths:
          - backend:
              serviceName: "apicurio-studio-ui"
              servicePort: 8093
            path: /
---
kind: Ingress
apiVersion: extensions/v1beta1
metadata:
  name: apicurio-studio-ws
  annotations:
  labels:
    module: apicurio-studio-ws
spec:
  rules:
    - host: apicurio.192.168.1.5.nip.io
      http:
        paths:
          - backend:
              serviceName: "apicurio-studio-ws"
              servicePort: 8092
            path: /designs
cemnura picture cemnura on 22 Jun 2020

Great news! I assume we need a separate PR for these changes? What do you think @t-rap and @jsenko ?

EricWittmann picture EricWittmann on 23 Jun 2020
👍2
Was this page helpful?
0 / 5 - 0 ratings

Related issues

0815fox picture 0815fox  ·  3Comments
0x218 picture 0x218  ·  5Comments
askfor picture askfor  ·  7Comments
AguRyan picture AguRyan  ·  3Comments
patrickceg picture patrickceg  ·  4Comments