Skip to content

Importing multiprocessing breaks _checkmodule in _pickle.c #120170

Closed
@li-dan

Description

@li-dan

Bug report

Bug description:

#86572 describes a bug where importing multiprocessing causes pickle to determine the wrong module for objects without __module__. #23403 fixed the bug, but only for the pure Python implementation of pickle. The C extension module is still broken.

Consider a file foo.py containing:

def f():
    pass
del f.__module__

Then, in an interactive session:

>>> import multiprocessing
>>> from foo import f
>>> import pickle
>>> import pickletools
>>> pickletools.dis(pickle.dumps(f))
    0: \x80 PROTO      4
    2: \x95 FRAME      21
   11: \x8c SHORT_BINUNICODE '__mp_main__'
   24: \x94 MEMOIZE    (as 0)
   25: \x8c SHORT_BINUNICODE 'f'
   28: \x94 MEMOIZE    (as 1)
   29: \x93 STACK_GLOBAL
   30: \x94 MEMOIZE    (as 2)
   31: .    STOP
highest protocol among opcodes = 4
>>> pickletools.dis(pickle._dumps(f))
    0: \x80 PROTO      4
    2: \x95 FRAME      13
   11: \x8c SHORT_BINUNICODE 'foo'
   16: \x94 MEMOIZE    (as 0)
   17: \x8c SHORT_BINUNICODE 'f'
   20: \x94 MEMOIZE    (as 1)
   21: \x93 STACK_GLOBAL
   22: \x94 MEMOIZE    (as 2)
   23: .    STOP
highest protocol among opcodes = 4

pickle.dumps tries to import f from __mp_main__. pickle._dumps correctly determines that f comes from the foo module.

CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux

Linked PRs

Activity

added
type-bugAn unexpected behavior, bug, or error
on Jun 6, 2024
added a commit that references this issue on Jun 6, 2024

pythongh-120170: Exclude __mp_main__ in C version of whichmodule()

added a commit that references this issue on May 23, 2025

gh-120170: Exclude __mp_main__ in C version of whichmodule() (#120171)

05a19b5

3 remaining items

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions

      Importing multiprocessing breaks _checkmodule in _pickle.c · Issue #120170 · python/cpython

      Follow Lee on X/Twitter - Father, Husband, Serial builder creating AI, crypto, games & web tools. We are friends :) AI Will Come To Life!

      Check out: eBank.nz (Art Generator) | Netwrck.com (AI Tools) | Text-Generator.io (AI API) | BitBank.nz (Crypto AI) | ReadingTime (Kids Reading) | RewordGame | BigMultiplayerChess | WebFiddle | How.nz | Helix AI Assistant