Skip to content

argparse.HelpFormatter subclasses may be broken by 3.14 changes #133653

Closed
@iritkatriel

Description

@iritkatriel

I came across this trying to run mypy benchmarks. The version I had contained

class AugmentedHelpFormatter(argparse.RawDescriptionHelpFormatter):
    def __init__(self, prog: str) -> None:
        super().__init__(prog=prog, max_help_position=30)

Which was rewritten in a newer version to:

class AugmentedHelpFormatter(argparse.RawDescriptionHelpFormatter):
    def __init__(self, prog: str, **kwargs: Any) -> None:
        super().__init__(prog=prog, max_help_position=30, **kwargs)

The changes in #124456 and #132323 can be made without breaking user code, if they accept the new args as **kwargs, extract their values from kwargs if they are there, and continue working as before if not. Is there a good reason not to do this?

Do these changes currently comply with Python's backwards compatibility policy?

Linked PRs

Activity

added
type-bugAn unexpected behavior, bug, or error
stdlibPython modules in the Lib dir
3.14bugs and security fixes
on May 8, 2025
iritkatriel

iritkatriel commented on May 8, 2025

@iritkatriel
MemberAuthor
hugovk

hugovk commented on May 8, 2025

@hugovk
Member

The changes in #124456 and #132323 can be made without breaking user code, if they accept the new args as **kwargs, extract their values from kwargs if they are there, and continue working as before if not. Is there a good reason not to do this?

Let's do this.

Would you like to open a PR? I'm also happy to do it.

iritkatriel

iritkatriel commented on May 8, 2025

@iritkatriel
MemberAuthor

Please go ahead.

iritkatriel

iritkatriel commented on May 8, 2025

@iritkatriel
MemberAuthor

Something like

        prefix_chars = kwargs.get('prefix_chars', '-')
        color = kwargs.get('color', False)
The-Compiler

The-Compiler commented on May 8, 2025

@The-Compiler
Contributor

Do these changes currently comply with Python's backwards compatibility policy?

FWIW, argparse.HelpFormatter itself is undocumented and the docstrings say:

cpython/Lib/argparse.py

Lines 62 to 64 in 0ec8fc8

(Also note that HelpFormatter and RawDescriptionHelpFormatter are only
considered public as object names -- the API of the formatter objects is
still considered an implementation detail.)

cpython/Lib/argparse.py

Lines 157 to 162 in 0ec8fc8

class HelpFormatter(object):
"""Formatter for generating usage messages and argument help strings.
Only the name of this class is considered a public API. All the methods
provided by the class are considered an implementation detail.
"""

But despite that, subclassing argparse.HelpFormatter seems to be a common occurrence, and many implementations seem to not use **kwargs.

hugovk

hugovk commented on May 8, 2025

@hugovk
Member

How does PR #133668 look?

serhiy-storchaka

serhiy-storchaka commented on May 8, 2025

@serhiy-storchaka
Member

I think this is a documentation issue. The only way to fix it reliably is to properly document the HelpFormatter class and how it cab be extended.

Currently the HelpFormatter class is not documented. It's subclassing is not officially supported. You need to look in the source code to find about max_help_position.

iritkatriel

iritkatriel commented on May 9, 2025

@iritkatriel
MemberAuthor

The only way to fix it reliably is to properly document the HelpFormatter class and how it cab be extended.

As discussed on the PR - doc change is welcome but it is not the only way to fix this.

I think we should add a setter method for the new fields so that the signature of __init__ doesn't need to change.

We saw two libraries break already, there will be more post release. There is no point repeatedly litigating this with users over the next 3-5 years if we can avoid that with a simple change now.

serhiy-storchaka

serhiy-storchaka commented on May 9, 2025

@serhiy-storchaka
Member

There is something similar issue #133745. It is more serious, because the documented contract was changed. For HelpFormatter there is no documented contract. We can apply solution similar to the code that was used before #128308 -- pass only prog to the factory, and set other parameters as private HelpFormatter attributes. This approach has many drawbacks. It still can break custom formatters, and for some users it will only defer breakage fr future versions. In long run, it is better to break the user code now. But we need the documentation to make sure that users will write correct code which will not break in future.

18 remaining items

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

3.14bugs and security fixesstdlibPython modules in the Lib dirtype-bugAn unexpected behavior, bug, or error

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions

    argparse.HelpFormatter subclasses may be broken by 3.14 changes · Issue #133653 · python/cpython

    Follow Lee on X/Twitter - Father, Husband, Serial builder creating AI, crypto, games & web tools. We are friends :) AI Will Come To Life!

    Check out: eBank.nz (Art Generator) | Netwrck.com (AI Tools) | Text-Generator.io (AI API) | BitBank.nz (Crypto AI) | ReadingTime (Kids Reading) | RewordGame | BigMultiplayerChess | WebFiddle | How.nz | Helix AI Assistant