Etherpad-lite: Performance degradation on 1.8.0 and newer versions

Cc @seballot

We have test coverage in for this afaik. See responsiveness frontend test.

The problem of the reflow is easier to realize when there are a large amount of nodes.
Is that one? https://github.com/ether/etherpad-lite/blob/develop/tests/frontend/specs/responsiveness.js
It's commented

Oh boy, I will uncomment and ensure we have working test coverage. Hopefully @seballot can find a performant fix hastily :)

thanks @JohnMcLear!

Uncommented, thanks for the really thorough and well put together bug report. Will bump @seballot again :+1:

Not heard back from @seballot yet despite email/cc here. I assume he's on vacation and will get onto this ASAP but if anyone else can take a stab while we wait that would be ideal :)

Hi guys ! thanks for this very clear bug report @joassouza, and sorry for the bug

I'm sorry I'm very focused on other project right now, I'll try to have a close look on this ticket on next week. But if anyone want to try to solve be my guest !!

@seballot just a bump for this coming week to ensure you have some time in your schedule to take a look at this please :) Thanks!

Hi !

I cannot reproduce, I made a 6000 lines long pad, and it works fine

Tested on Debian 9 -> Chromium 73 & Firefox 68

Why your line numbers looks bad (number 1 is displayed okay, then from number 2 it's too big and not aligned) ?
Could it be a problem in your local code?
Maybe the performance degradation is only on Mac? can someone else try please?

Haven't tested but can see Firefox responsive test fail that used to pass. Try Firefox responsive test with no-skin set to see if it passed?

test pass for me on both chromium and firefox

Yea it just passed for me too, so I need to dig deeper but I wonder if it passes for us because we're on fast(ish) pc's and sauce labs throws up some peasantry vm to handle the task?

I also experience zero lag on ff on ubuntu

Okay so changing amount to 1600000 (creates 8k lines) does make the lag noticeable for me.

Try doing that @seballot , tests/frontend/specs/responsiveness.js line 26

I can't confirm if this is a new bug though, it would be worth checking out 1.7.9 or whatever and ensuring that the behaviour is actually different. There must be a reason the test for responsiveness was at around the 1k lines mark.

It's also worth noting that the experience on Firefox 52 is the problem, modern FF seems fine? The responsiveness test crashes on FF 52 but it's fine in 80. In fact it crashes the browser. -- So I guess test in Firefox 52? I'm testing on SL now.

Responsive test fails on this.timeout is not a function error. https://github.com/ether/etherpad-lite/pull/4257/files

Just to give more context. The tests were made on Chrome on MacOs. We could reproduce the same error on really fast machines like i9, 64gb RAM.
@seballot could you try to edit in the beginning of the pad?
I've just tried on a Windows machine using chrome as well and I've got the same delay.
https://video.etherpad.com/p/example_long_pad

I have been testing Firefox like an idiot, will test Chrome now.

Okay wow yeah this is bad. I changed amount to 800000 and the lag is horrible.

I believe the delay increases if you edit in the beginning of the pad. Like, in the first line of it.

I'm not overly familiar with Chrome Performance tools but this is what I see when trying to edit the responsiveness.js pad.

Notice that I don't get all the layout time like you?

I've tried on FF on a Windows machine and I've got the same delay. Using this pad

https://video.etherpad.com/p/example_long_pad
About the reflow, it depends on which CSS is applied, browser, js code the triggers it, .... That's why I've done the tests without any plugin. I have to check the code of the responsiveness test to understand why in those tests you guys don't realize the delay.

Hi ! indeed with 8K it start freezing. But I switched to 1.7.5 and with the pad of 8k it freeze the same (I would even say it worst !)

Anyway, it's a fact that etherpad get slow when pads are too big. I don't feel it's a critical bug, because pad are not made to writes books, and I think most people use them for short text.

But yes probably we could improve. But I feel it will be a painful job :) Also I'm not sure it will be only related to standard layout trashing, because one specificity is that etherpad is using different iframes, and so we always need to run javascript to align the layout between each other (for the line height for example)

I would try to have a look, but I'm not promising anything because
1- it does not seems very funny :)
2- I don't feel it's a critical bug

@JohnMcLear I tried to have a look now, but I don't understand, every change I make to ace2_inner.js (even deleting the whole file) does not affect my local instance, it still working the same. What am I missing?

ace2_inner.js is requested without version string atm, so the URL to ace2_inner.js does not change and your browser cache is used. When you set maxAge=0 in settings.json and/or disable browser cache it should serve the new file without restart.

Hi @webzwo0i ! thanks for you help ! I finally remember that I need to invalidate myself the cache of ace2_inner because it's loaded in the iframe...

(maxAge=0 had no impact btw)

A bit late now, will look tomorrow !

One trick is to use network tab open as it disables caching.

@joassouza can you confirm Sebastians findings?

Ok done ! actually it was not very complicated, thanks to @joassouza who made all the work of finding both problem and solution

and I confirm the bug is not related to recent UI changes, it has been there for a long time I think !

I guess this can be closed now with the fix #4267
Thanks one more time @seballot