Skip to content

Free threading scaling bottleneck in sys.intern() #146041

@colesbury

Description

@colesbury

Bug report

Calling sys.intern() on a string that's not interned acquires the interpreter-wide interned_mutex. This can lead to scaling bottlenecks either due to explicit sys.intern() calls or indirectly because PyObject_SetAttr calls _PyUnicode_InternMortal.

We should avoid the lock acquisition if there is an interned copy already present in the interned_dict.

For example:

_ATTR_PREFIX = "bench"

@register_benchmark
def setattr_non_interned():
    prefix = _ATTR_PREFIX
    obj = MyObject()
    for _ in range(1000 * WORK_SCALE):
        setattr(obj, f"{prefix}_a", None)
        setattr(obj, f"{prefix}_b", None)
        setattr(obj, f"{prefix}_c", None)

This came up when looking at Django scaling bottlenecks.

Metadata

Metadata

Assignees

Labels

3.14bugs and security fixes3.15new features, bugs and security fixesperformancePerformance or resource usagetopic-free-threadingtype-bugAn unexpected behavior, bug, or error

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions