Lock contention inside `_PyType_LookupRef`

[nascheme]

Feature or enhancement

Proposal:

As part of adding free-threading support to LibCST, we noticed there is a lot of lock contention on TYPE_LOCK inside the _PyType_LookupRef function. In the LibCST, the common "visitor" pattern is used. For example, in the _visitors.py module there is the code:

visit_func = getattr(self, f"visit_{type(node).__name__}", None)

The second argument to getattr() is a non-interned string and it causes the cached and lock-free path of _PyType_LookupRef() never to be taken. Instead, the TYPE_LOCK mutex is acquired on each lookup. This obviously scales very badly if there are multiple threads looking up class methods using this pattern.

Testing was done with Python 3.13 but I believe the same issue exists with 3.14.

Has this already been discussed elsewhere?

This is a minor feature, which does not need previous discussion elsewhere

Links to previous discussion of this feature:

No response

Linked PRs

• [3.13] GH-132380: Avoid locking in _PyType_LookupRef() #132381
• gh-127266: avoid data races when updating type slots #131174
• [3.13] GH-132380: Add optimization for non-interned type lookup. #132651
• GH-132380: Add optimization for non-interned type lookup. #132652
• gh-132380: Use unicode hash/compare for the mcache. #133669
• GH-132380: Avoid locking in type lookup. #135112

[nascheme]

GH-127266 includes changes that effectively does the same thing as GH-132380 (avoid taking lock if the type version is already assigned).