CPython 3.12 embedded in WeeChat causes segfault on subsequent calls to Py_EndInterpreter

[trygveaa]

Crash report

What happened?

WeeChat embeds CPython in order to run Python scripts inside WeeChat. It can load multiple scripts and they each get their own interpreter. When a script is loaded Py_NewInterpreter is called, and when it's unloaded Py_EndInterpreter is called.

With CPython 3.12 loading two scripts and then unloading them in the same order causes a segmentation fault. Interestingly, the segmentation fault doesn't happen if the script that was loaded last is unloaded first.

I bisected this and found it was introduced in commit de64e75. I also noticed that the crash doesn't occur in the main branch, and did another bisect and found it was fixed in commit 7a7bce5.

This issue seems similar to the one reported in #115649 which is also introduced by the same commit, but that one still crashes on the main branch (commit 735fc2c).

I haven't been able to reproduce this outside of WeeChat unfortunately, but here is a backtrace from the crash with WeeChat, with commit de64e75 of CPython and commit ec56a1103f47b15a641ff93528fd6f50025dd524 of WeeChat.

Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x000074b5e5fd2700 in ?? ()
[Current thread is 1 (Thread 0x74b5e7bec940 (LWP 1881931))]
(gdb) bt
#0  0x000074b5e5fd2700 in ?? ()
#1  <signal handler called>
#2  0x000074b5e64f9ddd in _PyGCHead_SET_PREV (prev=<optimized out>, gc=<optimized out>) at ./Include/internal/pycore_gc.h:74
#3  _PyObject_GC_UNTRACK (op=0x74b5dfb746d0) at ./Include/internal/pycore_object.h:228
#4  PyObject_GC_UnTrack (op_raw=op_raw@entry=0x74b5dfb746d0) at Modules/gcmodule.c:2241
#5  0x000074b5e63c430c in module_dealloc (m=0x74b5dfb746d0) at Objects/moduleobject.c:672
#6  0x000074b5e63c393d in Py_DECREF (op=<optimized out>) at ./Include/object.h:681
#7  Py_XDECREF (op=<optimized out>) at ./Include/object.h:777
#8  meth_dealloc (m=0x74b5dfb81210) at Objects/methodobject.c:170
#9  0x000074b5e63b5d00 in Py_DECREF (op=0x74b5dfb81210) at ./Include/object.h:681
#10 Py_XDECREF (op=0x74b5dfb81210) at ./Include/object.h:777
#11 insertdict (interp=0x74b5e0e42010, mp=mp@entry=0x74b5df1d3d40, key=0x74b5dfb7db30, hash=<optimized out>, value=value@entry=0x74b5e6750240 <_Py_NoneStruct>) at Objects/dictobject.c:1304
#12 0x000074b5e63b6107 in _PyDict_SetItem_Take2 (value=0x74b5e6750240 <_Py_NoneStruct>, key=<optimized out>, mp=0x74b5df1d3d40) at Objects/dictobject.c:1854
#13 0x000074b5e63c5684 in _PyModule_ClearDict (d=0x74b5df1d3d40) at Objects/moduleobject.c:619
#14 0x000074b5e63c5a6e in _PyModule_Clear (m=m@entry=0x74b5df1e12b0) at Objects/moduleobject.c:567
#15 0x000074b5e64c884e in finalize_modules_clear_weaklist (verbose=0, weaklist=0x74b5dfbed080, interp=0x74b5e0e42010) at Python/pylifecycle.c:1491
#16 finalize_modules (tstate=tstate@entry=0x74b5e0ea0400) at Python/pylifecycle.c:1574
#17 0x000074b5e64cc476 in Py_EndInterpreter (tstate=0x74b5e0ea0400) at Python/pylifecycle.c:2137
#18 0x000074b5e6a5bce6 in weechat_python_unload (script=0x5d9a7b80ee60) at /home/trygve/dev/weechat/src/plugins/python/weechat-python.c:947
#19 0x000074b5e6a5bea6 in weechat_python_unload_all () at /home/trygve/dev/weechat/src/plugins/python/weechat-python.c:996
#20 0x000074b5e7b81cea in plugin_script_end (weechat_plugin=0x5d9a7b29ad50, plugin_data=0x74b5e6a9e640 <python_data>) at /home/trygve/dev/weechat/src/plugins/plugin-script.c:1841
#21 0x000074b5e6a5d8c4 in weechat_plugin_end (plugin=0x5d9a7b29ad50) at /home/trygve/dev/weechat/src/plugins/python/weechat-python.c:1634
#22 0x00005d9a78ed4c38 in plugin_unload (plugin=0x5d9a7b29ad50) at /home/trygve/dev/weechat/src/plugins/plugin.c:1261
#23 0x00005d9a78ed4d9f in plugin_unload_all () at /home/trygve/dev/weechat/src/plugins/plugin.c:1313
#24 0x00005d9a78ed50f8 in plugin_end () at /home/trygve/dev/weechat/src/plugins/plugin.c:1433
#25 0x00005d9a78e04340 in weechat_end (gui_end_cb=0x5d9a78ecb8ae <gui_main_end>) at /home/trygve/dev/weechat/src/core/weechat.c:709
#26 0x00005d9a78e03163 in main (argc=4, argv=0x7ffdb171c178) at /home/trygve/dev/weechat/src/gui/curses/normal/main.c:45

This was produced by creating these two python scripts:

dummy1.py:

import weechat

if weechat.register("dummy1", "trygveaa", "0.1", "MIT", "Dummy script 1", ""):
    weechat.prnt("", "Loaded dummy script 1")

dummy2.py

import weechat

if weechat.register("dummy2", "trygveaa", "0.1", "MIT", "Dummy script 2", ""):
    weechat.prnt("", "Loaded dummy script 2")

And then running weechat -t -r '/script load dummy1.py; /script load dummy2.py; /quit'.

Also, here is the issue report for WeeChat: weechat/weechat#2046

Since it's fixed in main it seems there won't be a problem with 3.13, but I wonder if the fix can be backported to 3.12?

CPython versions tested on:

3.12, CPython main branch

Operating systems tested on:

Linux

Output from running 'python -VV' on the command line:

Python 3.12.0a7+ (tags/v3.12.0a7-340-gde64e75616:de64e75616, Mar 8 2024, 19:43:39) [GCC 13.2.1 20230801]

Linked PRs

• [3.12] gh-116510: Fix crash during sub-interpreter shutdown #124536
• [3.12] gh-116510: Fix a crash due to shared immortal interned strings. #124541
• gh-116510: Fix crash during sub-interpreter shutdown #124645
• gh-116510: Fix crash due to shared immortal interned strings. #124646
• [3.13] gh-116510: Fix crash due to shared immortal interned strings. (gh-124646) #124648
• [3.13] gh-116510: Fix crash during sub-interpreter shutdown (gh-124645) #124649
• gh-116510: Fix a Crash Due to Shared Immortal Interned Strings #124865
• [3.13] gh-116510: Fix a Crash Due to Shared Immortal Interned Strings (gh-124865) #125204
• [3.12] gh-116510: Fix a Crash Due to Shared Immortal Interned Strings (gh-124865) #125205

[ericsnowcurrently]

As to backporting 7a7bce5 (gh-113412), it wasn't obvious at the time that it was worth backporting, relative to the complexity of the change. Ultimately, that's a call for the 3.12 release manager, @Yhg1s to make.

CC @nascheme

[mooninite]

I tried unsuccessfully backporting 7a7bce5 as threading tests failed. If someone could attach a 3.12 backport patch I could test Kodi.

[nascheme]

I have a backport of the patch mostly done. I can finish it and then you can test. I think backporting this change would be a good idea given that it seems to work without issue in 3.13 and would solve a few problems for users of Python 3.12.

[ericsnowcurrently]

ping @Yhg1s