Apicurio-studio: Saving Api Changes Problem while Collaborating

That's not good. What version of Apicurio are you using? Is this happening in the cloud version or are you running it locally? I have somewhat recently found and fixed a problem just like this (but only when using the SHARE_FOR_EVERYONE feature), but if you're using the latest version or not using that feature then this might be a new bug.

We have the latest version 0.243.final running in kubernetes and SHARE_FOR_EVERYONE is activated.

Thanks for getting back to us so quickly @EricWittmann.
Curiously the changes are shown in the Activity Log even though they are not persistent after a reload.
Hopefully @t-rap can have a look at the database soon.

Hey,

Sorry for the late response:

We're running apicurio-studio on kubernetes with the docker images you provide through the docker hub.
So
Apicurio-studio-ui, -ws, -api, -db (image: percona:5.7)

When our Devs use it, the debug message on apicurio-studio-api is the following:

2019-11-20 15:12:05,053 DEBUG [io.apicurio.hub.core.storage.jdbc.JdbcStorage] (default task-66) Selecting the most recent api_content row of type 'document' for: 12 2019-11-20 15:12:05,056 DEBUG [io.apicurio.hub.core.storage.jdbc.JdbcStorage] (default task-66) Inserting an Editing Session UUID row: REDACTED 2019-11-20 15:12:05,070 DEBUG [io.apicurio.hub.api.rest.impl.DesignsResource] (default task-66) Created Session ID: REDACTED 2019-11-20 15:12:05,070 DEBUG [io.apicurio.hub.api.rest.impl.DesignsResource] (default task-66) Secret: REDACTED 2019-11-20 15:12:14,973 DEBUG [io.apicurio.hub.api.rest.impl.DesignsResource] (default task-66) Retrieving contributors list for design with ID: 12 2019-11-20 15:12:14,973 DEBUG [io.apicurio.hub.core.storage.jdbc.JdbcStorage] (default task-66) Selecting all contributors for API Design: 12 2019-11-20 15:12:14,973 DEBUG [io.apicurio.studio.shared.config.Configuration] (default task-65) Config Property: APICURIO_SHARE_FOR_EVERYONE/apicurio.share.for.everyone = true 2019-11-20 15:12:14,973 DEBUG [io.apicurio.hub.core.storage.jdbc.JdbcStorage] (default task-65) Selecting activity for API Design: 12 from 0 to 10 2019-11-20 15:12:14,983 DEBUG [io.apicurio.hub.core.storage.jdbc.JdbcStorage] (default task-64) Selecting mock activity for API Design: 12 from 0 to 20 2019-11-20 15:12:14,997 DEBUG [io.apicurio.hub.api.rest.impl.DesignsResource] (default task-67) Getting an API design with ID 12 2019-11-20 15:12:14,997 DEBUG [io.apicurio.hub.core.storage.jdbc.JdbcStorage] (default task-67) Selecting a single API Design: 12 2019-11-20 15:12:17,810 DEBUG [io.apicurio.hub.api.rest.impl.DesignsResource] (default task-67) Getting an API design with ID 12 2019-11-20 15:12:17,810 DEBUG [io.apicurio.hub.core.storage.jdbc.JdbcStorage] (default task-67) Selecting a single API Design: 12 2019-11-20 15:12:17,860 DEBUG [io.apicurio.hub.api.rest.impl.DesignsResource] (default task-67) Editing an API Design with ID 12 2019-11-20 15:12:17,860 DEBUG [io.apicurio.hub.api.rest.impl.DesignsResource] (default task-67) USER: REDACTED 2019-11-20 15:12:17,860 DEBUG [io.apicurio.hub.core.storage.jdbc.JdbcStorage] (default task-67) Selecting the most recent api_content row of type 'document' for: 12 2019-11-20 15:12:17,862 DEBUG [io.apicurio.hub.core.storage.jdbc.JdbcStorage] (default task-67) Inserting an Editing Session UUID row: REDACTED 2019-11-20 15:12:17,875 DEBUG [io.apicurio.hub.api.rest.impl.DesignsResource] (default task-67) Created Session ID: REDACTED 2019-11-20 15:12:17,875 DEBUG [io.apicurio.hub.api.rest.impl.DesignsResource] (default task-67) Secret: REDACTED

For apicurio-studio-ws gives:

2019-11-20 15:14:31,640 DEBUG [io.apicurio.hub.core.editing.ops.processors.CommandProcessor] (default task-3) user: REDACTED 2019-11-20 15:14:31,640 DEBUG [io.apicurio.hub.core.storage.jdbc.JdbcStorage] (default task-3) Inserting a 'command' content row for: 12 2019-11-20 15:14:31,659 DEBUG [io.apicurio.hub.core.editing.ops.processors.CommandProcessor] (default task-3) ACK sent back to client. 2019-11-20 15:14:31,659 DEBUG [io.apicurio.hub.core.editing.ops.processors.CommandProcessor] (default task-3) Command propagated to 'other' clients. 2019-11-20 15:14:33,145 DEBUG [io.apicurio.hub.core.editing.ops.processors.OperationProcessorDispatcher] (default task-3) Received a "ping" message/operation from a client for API Design: 12 2019-11-20 15:14:44,989 DEBUG [io.apicurio.hub.core.editing.ops.processors.OperationProcessorDispatcher] (default task-3) Received a "ping" message/operation from a client for API Design: 15 2019-11-20 15:14:46,691 DEBUG [io.apicurio.hub.editing.EditApiDesignEndpoint] (default task-3) Closing a WebSocket due to: 2019-11-20 15:14:46,692 DEBUG [io.apicurio.hub.editing.EditApiDesignEndpoint] (default task-3) designId: 12

So this was definitely the bug I fixed recently - it's possible that I fixed it after the most recent release. Let me check that and get back to you here.

OK I have confirmed that my fix for this problem was done after the most recent release. So you have two options to get this fixed:

1) Wait for the next release
2) Build latest from master and run that

Option (2) is probably more accessible than you might think, but still obviously not ideal for you. I could do another release the end of the week probably. There are a couple of experimental features that have been added which I need to disable (or at least mark as such in the UI) before I can trigger a release.

I'm kicking off a new release right now - will include the fix for this issue. Marking as closed so it's included in the release notes. :)

hi Eric,

The issue seems to still be there, but it throws an exception now. I'm unsure if it is our fault or not. Do you have any hint where i can deeper look into it?

```2019-11-22 10:16:14,057 ERROR [io.apicurio.hub.core.storage.jdbc.JdbcStorage] (default task-1) Error getting latest content document: java.lang.IllegalStateException: Expected one element, but found none
at org.jdbi.v3.core.result.ResultIterable.one(ResultIterable.java:135)
at io.apicurio.hub.core.storage.jdbc.JdbcStorage.lambda$getLatestContentDocument$23(JdbcStorage.java:655)
at org.jdbi.v3.core.Jdbi.withHandle(Jdbi.java:341)
at io.apicurio.hub.core.storage.jdbc.JdbcStorage.getLatestContentDocument(JdbcStorage.java:648)
at io.apicurio.hub.core.storage.jdbc.JdbcStorage$Proxy$_$$_WeldClientProxy.getLatestContentDocument(Unknown Source)
at io.apicurio.hub.core.storage.RollupExecutor.rollupCommands(RollupExecutor.java:56)
at io.apicurio.hub.core.storage.RollupExecutor.rollupCommands(RollupExecutor.java:108)
at io.apicurio.hub.core.storage.RollupExecutor$Proxy$_$$_WeldClientProxy.rollupCommands(Unknown Source)
at io.apicurio.hub.core.editing.EditingSession.close(EditingSession.java:131)
at io.apicurio.hub.core.editing.EditingSessionManager.closeEditingSession(EditingSessionManager.java:116)
at io.apicurio.hub.core.editing.EditingSessionManager$Proxy$_$$_WeldClientProxy.closeEditingSession(Unknown Source)
at io.apicurio.hub.editing.EditApiDesignEndpoint.onCloseSession(EditApiDesignEndpoint.java:215)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at io.undertow.websockets.jsr.annotated.BoundMethod.invoke(BoundMethod.java:87)
at io.undertow.websockets.jsr.annotated.AnnotatedEndpoint$4.run(AnnotatedEndpoint.java:201)
at io.undertow.websockets.jsr.ServerWebSocketContainer$1.call(ServerWebSocketContainer.java:169)
at io.undertow.websockets.jsr.ServerWebSocketContainer$1.call(ServerWebSocketContainer.java:166)
at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43)
at org.wildfly.extension.undertow.security.SecurityContextThreadSetupAction.lambda$create$0(SecurityContextThreadSetupAction.java:105)
at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1502)
at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1502)
at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1502)
at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1502)
at io.undertow.websockets.jsr.ServerWebSocketContainer.invokeEndpointMethod(ServerWebSocketContainer.java:603)
at io.undertow.websockets.jsr.ServerWebSocketContainer$6.run(ServerWebSocketContainer.java:589)
at io.undertow.websockets.jsr.OrderedExecutor$ExecutorTask.run(OrderedExecutor.java:67)
at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35)
at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1985)
at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1487)
at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1378)
at java.lang.Thread.run(Thread.java:748)

2019-11-22 10:16:14,057 ERROR [io.apicurio.hub.core.editing.EditingSession] (default task-1) Error detected closing an Editing Session.: io.apicurio.hub.core.exceptions.NotFoundException
at io.apicurio.hub.core.storage.jdbc.JdbcStorage.getLatestContentDocument(JdbcStorage.java:659)
at io.apicurio.hub.core.storage.jdbc.JdbcStorage$Proxy$_$$_WeldClientProxy.getLatestContentDocument(Unknown Source)
at io.apicurio.hub.core.storage.RollupExecutor.rollupCommands(RollupExecutor.java:56)
at io.apicurio.hub.core.storage.RollupExecutor.rollupCommands(RollupExecutor.java:108)
at io.apicurio.hub.core.storage.RollupExecutor$Proxy$_$$_WeldClientProxy.rollupCommands(Unknown Source)
at io.apicurio.hub.core.editing.EditingSession.close(EditingSession.java:131)
at io.apicurio.hub.core.editing.EditingSessionManager.closeEditingSession(EditingSessionManager.java:116)
at io.apicurio.hub.core.editing.EditingSessionManager$Proxy$_$$_WeldClientProxy.closeEditingSession(Unknown Source)
at io.apicurio.hub.editing.EditApiDesignEndpoint.onCloseSession(EditApiDesignEndpoint.java:215)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at io.undertow.websockets.jsr.annotated.BoundMethod.invoke(BoundMethod.java:87)
at io.undertow.websockets.jsr.annotated.AnnotatedEndpoint$4.run(AnnotatedEndpoint.java:201)
at io.undertow.websockets.jsr.ServerWebSocketContainer$1.call(ServerWebSocketContainer.java:169)
at io.undertow.websockets.jsr.ServerWebSocketContainer$1.call(ServerWebSocketContainer.java:166)
at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43)
at org.wildfly.extension.undertow.security.SecurityContextThreadSetupAction.lambda$create$0(SecurityContextThreadSetupAction.java:105)
at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1502)
at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1502)
at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1502)
at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1502)
at io.undertow.websockets.jsr.ServerWebSocketContainer.invokeEndpointMethod(ServerWebSocketContainer.java:603)
at io.undertow.websockets.jsr.ServerWebSocketContainer$6.run(ServerWebSocketContainer.java:589)
at io.undertow.websockets.jsr.OrderedExecutor$ExecutorTask.run(OrderedExecutor.java:67)
at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35)
at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1985)
at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1487)
at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1378)
at java.lang.Thread.run(Thread.java:748)```

I'm pretty sure this is exactly the error I would expect if the bug fix wasn't present. Let me dig up some links to the code...

ok, i just checked to not waste your time, but it seems to be the final version:

Version | 0.2.44.Final
-- | --
Built On | Nov 21, 2019
Edition | CLIv7
Project URL | http://www.apicur.io/

We tested old APIs which existed already, created new APIs and tested and we tried it with different Accounts.

OK thanks. So what's happening with that stack trace is that, whenever all editors of an API are done editing (leave/close the editor) the server will try to rollup all of the changes into a document snapshot. This is done by a rollup executor here:

https://github.com/Apicurio/apicurio-studio/blob/master/back-end/hub-core/src/main/java/io/apicurio/hub/core/storage/RollupExecutor.java

It's trying to do this:

1) Get the LAST document snapshot
2) Get a list of all changes (commands) since then
3) Apply all changes/commands to the last document snapshot
4) Store the NEW snapshot back to the DB

It's failing on step #1 because it can't find the previous snapshot. This should be impossible because a snapshot is created when the API design is first created. However, it's happening probably because the "share for everyone" option is enabled for the -api Apicurio component but not for the -ws Apicurio component. So the original snapshot exists in the table but was created by User A, so when User B tries to do the rollup (and share-for-everyone is disabled) then the wrong SQL gets executed when looking for the snapshot and it's not found. So it's likely that this code is not working properly due to a mis-configuration of the share-for-everyone feature only in the -ws apicurio component:

https://github.com/Apicurio/apicurio-studio/blob/master/back-end/hub-core/src/main/java/io/apicurio/hub/core/storage/jdbc/JdbcStorage.java#L650-L655

If share-for-everyone is enabled in -api but not is -ws then this will occur.

I'd need to see your kubernetes setup (side note: any chance you'd like to contribute that?) but I'm guessing that if you based it on the docker compose config, you're missing the share-for-everyone config option on the -ws component. Here is one of the key fixes for this issue:

https://github.com/Apicurio/apicurio-studio/commit/7f4994bc907e1720ffd6f8ff81e844c032edbf79#diff-b1ff2c3381198f745ae9dc8add793d61

Hey Eric,

thanks for the hint. We "fixed" our problem by setting share-for-everyone also onto -ws pods. I didn't see it as an option on https://hub.docker.com/r/apicurio/apicurio-studio-ws. That's why i didn't put it in there.

If you want, i would clean up the k8s-configs and send you a pull request with it. I think i can get that done till the beginning of next week.

Great! I'll update the docs for that image to include that option. No rush on the k8s configs, but the contribution would be very welcome.