Hexchat: core dumped on Arch Linux with lua

Note that Lua 5.4.3 is a part of Arch' testing repository. And it is where you see the issue.

The stable Arch repo still uses lua 5.4.2.

Indeed, it seems with lua 5.4.2 everything works fine, with lua 5.4.3 it's crashing.

Btw does hexchat recompilation for the new lua help?

No, I've tried that and it didn't help. Also just downgrading lua on my machine makes it work so I suspect a bug in lua then.

I've tried bisecting that, but failed miserably, rebuilding lua on Arch from source instead of release archive is painful :disappointed:

Here is the dump info produced with non-stripped lua library:

Stack trace of thread 119974:
#0  0x00007f038bbd3ef5 raise (libc.so.6 + 0x3cef5)
#1  0x00007f038bbbd862 abort (libc.so.6 + 0x26862)
#2  0x00007f0388b73026 luaD_throw.cold (liblua.so.5.4 + 0x8026)
#3  0x00007f0388b7a2ca luaG_errormsg (liblua.so.5.4 + 0xf2ca)
#4  0x00007f0388b7a3e4 luaG_runerror (liblua.so.5.4 + 0xf3e4)
#5  0x00007f0388b7a461 luaG_typeerror (liblua.so.5.4 + 0xf461)
#6  0x00007f0388b7a542 luaG_callerror (liblua.so.5.4 + 0xf542)
#7  0x00007f0388b7b1bb luaD_tryfuncTM (liblua.so.5.4 + 0x101bb)
#8  0x00007f0388b7b632 luaD_precall (liblua.so.5.4 + 0x10632)
#9  0x00007f0388b7ba17 luaD_callnoyield (liblua.so.5.4 + 0x10a17)
#10 0x00007f0388b7cb45 luaF_close (liblua.so.5.4 + 0x11b45)
#11 0x00007f0388b7358d lua_settop (liblua.so.5.4 + 0x858d)
#12 0x00007f0388cb0d49 n/a (lua.so + 0x4d49)
#13 0x00007f0388cb336b hexchat_plugin_init (lua.so + 0x736b)
#14 0x0000559f75d1f8c0 plugin_add (hexchat + 0x8d8c0)
#15 0x0000559f75d1f9ca plugin_load (hexchat + 0x8d9ca)
#16 0x0000559f75d1fc2b n/a (hexchat + 0x8dc2b)
#17 0x0000559f75d2743e for_files (hexchat + 0x9543e)
#18 0x0000559f75d2032e plugin_auto_load (hexchat + 0x8e32e)
#19 0x0000559f75d0b069 new_ircwindow (hexchat + 0x79069)
#20 0x0000559f75d2965b servlist_connect (hexchat + 0x9765b)
#21 0x0000559f75ced893 n/a (hexchat + 0x5b893)
#22 0x00007f038c97dddf g_closure_invoke (libgobject-2.0.so.0 + 0x12ddf)
#23 0x00007f038c9a6fbd n/a (libgobject-2.0.so.0 + 0x3bfbd)
#24 0x00007f038c99acad g_signal_emit_valist (libgobject-2.0.so.0 + 0x2fcad)
#25 0x00007f038c99b210 g_signal_emit (libgobject-2.0.so.0 + 0x30210)
#26 0x00007f038c45d596 n/a (libgtk-x11-2.0.so.0 + 0x88596)
#27 0x00007f038c97dddf g_closure_invoke (libgobject-2.0.so.0 + 0x12ddf)
#28 0x00007f038c9a6c2c n/a (libgobject-2.0.so.0 + 0x3bc2c)
#29 0x00007f038c99acad g_signal_emit_valist (libgobject-2.0.so.0 + 0x2fcad)
#30 0x00007f038c99b210 g_signal_emit (libgobject-2.0.so.0 + 0x30210)
#31 0x00007f038c45c46a n/a (libgtk-x11-2.0.so.0 + 0x8746a)
#32 0x00007f038c50b0a8 n/a (libgtk-x11-2.0.so.0 + 0x1360a8)
#33 0x00007f038c97dddf g_closure_invoke (libgobject-2.0.so.0 + 0x12ddf)
#34 0x00007f038c9a69cb n/a (libgobject-2.0.so.0 + 0x3b9cb)
#35 0x00007f038c99a2db g_signal_emit_valist (libgobject-2.0.so.0 + 0x2f2db)
#36 0x00007f038c99b210 g_signal_emit (libgobject-2.0.so.0 + 0x30210)
#37 0x00007f038c630275 n/a (libgtk-x11-2.0.so.0 + 0x25b275)
#38 0x00007f038c5096d6 gtk_propagate_event (libgtk-x11-2.0.so.0 + 0x1346d6)
#39 0x00007f038c509b4b gtk_main_do_event (libgtk-x11-2.0.so.0 + 0x134b4b)
#40 0x00007f038c3783be n/a (libgdk-x11-2.0.so.0 + 0x5e3be)
#41 0x00007f038c889f9c g_main_context_dispatch (libglib-2.0.so.0 + 0x53f9c)
#42 0x00007f038c8dda49 n/a (libglib-2.0.so.0 + 0xa7a49)
#43 0x00007f038c889503 g_main_loop_run (libglib-2.0.so.0 + 0x53503)
#44 0x00007f038c5089fe gtk_main (libgtk-x11-2.0.so.0 + 0x1339fe)
#45 0x0000559f75ccd4da fe_main (hexchat + 0x3b4da)
#46 0x0000559f75cbf828 main (hexchat + 0x2d828)
#47 0x00007f038bbbeb25 __libc_start_main (libc.so.6 + 0x27b25)
#48 0x0000559f75cbf99e _start (hexchat + 0x2d99e)

My lua bisecting points to this lua commit as the root of the crash https://github.com/lua/lua/commit/511d53a826760dd11cd82947184583e2d094e2d2

One thing that happened in this commit is that lua started enforcing 5.4.0 compat mode and following compatibility behavior has been dropped. I wonder if it is something that needs to be adjusted at hexchat side.

For compatibility reasons,
this function may close slots marked as to-be-closed @see{lua_toclose},
and therefore it can run arbitrary code.
You should not rely on this behavior:
Instead, always close to-be-closed slots explicitly,
with @Lid{lua_closeslot}, before removing them from the stack.

cc @culb who might have some ideas what how this change affects hexchat plugin.

@invidian Lua makefile does not provide a way to build *.so (used at Arch) so for bisecting I had to build *.so manually. Here is a way how I ran my bisecting:

cd lua-sources
# loop
git clean -xdf
make MYCFLAGS="-fPIC" -j
cc -shared -ldl -o liblua.so.5.4.3 lapi.o lcode.o lctype.o ldebug.o ldo.o ldump.o lfunc.o lgc.o llex.o \
        lmem.o lobject.o lopcodes.o lparser.o lstate.o lstring.o ltable.o \
        ltm.o lundump.o lvm.o lzio.o ltests.o lbaselib.o ldblib.o liolib.o lmathlib.o loslib.o ltablib.o lstrlib.o \
        lutf8lib.o loadlib.o lcorolib.o linit.o lauxlib.o
sudo mv liblua.so.5.4.3 /usr/lib/liblua.so.5.4.3
# test hexchat here
git bisect (good|bad)
# goto beginning of the loop

Here is my bisect log

git bisect start
# bad: [eadd8c7178c79c814ecca9652973a9b9dd4cc71b] Added option LUA_NOBUILTIN
git bisect bad eadd8c7178c79c814ecca9652973a9b9dd4cc71b
# good: [9d067ab73b6befa0a5418f1df35c711f6c6918b3] Optimization for 'n^2'
git bisect good 9d067ab73b6befa0a5418f1df35c711f6c6918b3
# good: [1f81baffadad9d955b030a1a29b9b06042a66552] Janitorial work
git bisect good 1f81baffadad9d955b030a1a29b9b06042a66552
# good: [5205f073c57ae4b69e90d35c02e3a1a1cca44eb4] Don't use tointegerns when luaV_tointegerns will do
git bisect good 5205f073c57ae4b69e90d35c02e3a1a1cca44eb4
# good: [b7eb21c1efbd33affb87479fc6055914fe9ab009] Normalization of metamethod typography in the manual
git bisect good b7eb21c1efbd33affb87479fc6055914fe9ab009
# bad: [a7b8b27dd39f45b9464ffc4226b0616c3ffe5ad7] Uses of "likely" in macros active to all users
git bisect bad a7b8b27dd39f45b9464ffc4226b0616c3ffe5ad7
# good: [f5df7f91f70234850484d26caf24e71e001e5304] Wrong assertion in 'getbaseline'
git bisect good f5df7f91f70234850484d26caf24e71e001e5304
# bad: [511d53a826760dd11cd82947184583e2d094e2d2] lua_settop/lua_pop closes to-be-closed variables
git bisect bad 511d53a826760dd11cd82947184583e2d094e2d2
# first bad commit: [511d53a826760dd11cd82947184583e2d094e2d2] lua_settop/lua_pop closes to-be-closed variables

Cross-posting https://bugs.archlinux.org/task/70213 reply to @invidian

I would not call it "lua regression". From my findings above Lua 5.4.3 has dropped lua 5.3 compatibility behavior. And Lua developer(s) are known for their aggressive language evolution without focusing too much on the backward compatibility.

So if it is confirmed that the reason for this crash is broken compatibility then there are 2 choices:

• fix hexchat and make it compatible with 5.4
• compile hexchat with lua 5.3 https://archlinux.org/packages/extra/x86_64/lua53/

It might be worth checking with Lua upstream about this change to check how users expect to deal with this compatibility breakage.

It seems following patch make it work for both lua 5.4.2 and 5.4.3:

diff --git plugins/lua/lua.c plugins/lua/lua.c
index d73fbb23..681576cd 100644
--- plugins/lua/lua.c
+++ plugins/lua/lua.c
@@ -1193,7 +1193,6 @@ static void patch_clibs(lua_State *L)
                }
                lua_pop(L, 1);
        }
-       lua_pop(L, 1);
 }

 static GPtrArray *scripts;

I was not able to trigger if(lua_type(L, -2) == LUA_TLIGHTUSERDATA && lua_type(L, -1) == LUA_TTABLE) block to see if we need to call lua_pop before break.

Indeed there seem to be too many pops. This should be the correct implementation:

static void patch_clibs(lua_State *L)
{
    lua_pushnil(L); /* push nil key to start iteration */
    while(lua_next(L, LUA_REGISTRYINDEX)) /* pop key. push key and value if ret != 0 */
    {
        if(lua_type(L, -2) == LUA_TLIGHTUSERDATA && lua_type(L, -1) == LUA_TTABLE)
        {
            lua_setfield(L, LUA_REGISTRYINDEX, "_CLIBS"); /* pop value */
                lua_pop(L, 1); /* pop key */
            break;
        }
        lua_pop(L, 1); /* pop value */
    }
}

Created #2559, as far as I understand, it's a bug exposed by latest version of Lua, which should be fixed.

@anatol interesting. My bisecting pointed at lua/lua@81c6021fb40a254d9a586b0cb53453bba8973d80 and it seems reverting either this commit or lua/lua@511d53a you pointed fixes the issue.