From 59fd3d4c7016d514ab3d15ef6f18a1ef9dc36f51 Mon Sep 17 00:00:00 2001 From: Peter Bierma Date: Sat, 1 Aug 2026 10:46:53 -0400 Subject: [PATCH 1/4] PEP 842: Address feedback from first discussion round --- peps/pep-0842.rst | 277 +++++++++++++++++++++++++++++++++------------- 1 file changed, 198 insertions(+), 79 deletions(-) diff --git a/peps/pep-0842.rst b/peps/pep-0842.rst index 9fb923db2da..02f453918d2 100644 --- a/peps/pep-0842.rst +++ b/peps/pep-0842.rst @@ -14,7 +14,7 @@ Abstract ======== This PEP proposes an ``__export__`` variable that modules can define to -limit visibility and access to variables from outside the module. +express intent behind visibility to variables from outside the module. For example: @@ -33,25 +33,37 @@ For example: .. code-block:: pycon >>> import spam + >>> 'Public' in dir(spam) + True + >>> 'Private' in dir(spam) + False >>> spam.Public >>> spam.Private - Traceback (most recent call last): - File "", line 1, in - spam.Private - ImportError: 'Private' is not exported by 'spam' + :1: RuntimeWarning: 'Private' is not exported by 'spam' + + + +This is **not** intended to be an access modifier for Python; see +:ref:`pep-842-not-an-access-modifier`. Motivation ========== -Private names can be difficult to disambiguate on their own ------------------------------------------------------------ +Module names need privacy +------------------------- + +A developer is writing a Python module. The module is intended to have one +"public" class -- a class that is intended for users of the module -- called +``PublicAPI``. As part of implementing ``PublicAPI``, the developer wants to +create another class, called ``Helper``. However, ``Helper`` is not meant to +be public in the same way that ``PublicAPI`` is public. ``Helper`` is supposed +to only be used by the developer of the module -- a "private" API. + +Nonetheless, the developer declares the two classes as such: -Imagine that a developer wants to define a private class in -their module. Their first instinct might be to simply define their class -as such: .. code-block:: python @@ -59,18 +71,22 @@ as such: class Helper: ... -However, this comes with no indication that ``Helper`` is supposed to be -internal to the module, so users may accidentally begin using and relying -on it. In fact, Python's interactive :func:`help` function will even include -``Helper`` in its output next to everything else in the module. + class PublicAPI: + ... + + +The problem with this is that ``Helper`` comes with no indication that it's not +a public API. It shows up in autocomplete by language servers, the :func:`dir` +function, Python's interactive :func:`help` function, and every other API meant +for introspection. How are users supposed to know that they aren't supposed to +use this? -Prefixed names are less maintainable ------------------------------------- +Prefixed names aren't necessarily a great solution +-------------------------------------------------- -When developing a Python module, it is common to prefix a name with ``_`` -to denote that it is private. So, as a solution to the above problem, -the developer prefixes the name with ``_``: +In Python, the convention for declaring private names is to prefix it +with ``_``. So, the developer changes ``Helper`` into ``_Helper``: .. code-block:: python @@ -78,13 +94,29 @@ the developer prefixes the name with ``_``: class _Helper: ... +This is generally the standard for Python libraries today, but it's not clear +that this is the best long term solution. This works (with some caveats; see the +sections below), but this is (subjectively) less readable, and does require +more keystrokes by the maintainer. Ideally, users shouldn't be tempted to +reach for private names from modules in the first place. -Now, it's clear to users that the name is internal, at the expense of the name -being (subjectively) less readable and requiring more keystrokes by the maintainer. +However, it is acknowledged that this idea is going against 30 years of +convention; even if this PEP is accepted, it's expected that "underscored" +names (names prefixed with a leading ``_``) will remain a staple of Python +for years to come. The purpose of this PEP is not to eliminate the need for +``_`` in module-level names, but instead to clear up corner cases where a +private name is ambigious or tempting. In other words, this PEP is intended +to improve expressiveness and clarity with private APIs, *not* to add brand +new functionality. -In addition, it can be difficult to remember where names need to be prefixed. -To put this issue into perspective, imagine that a developer wants to import -some other modules in their code: + +It's not always clear where names need prefixing +************************************************ + +Python defines names through many different constructs, some of which are not +always clear or intuitive to the developer. As a result, it can be difficult to +remember where names need to be prefixed. To put this issue into perspective, +imagine that a developer wants to import some other modules in their code: .. code-block:: python @@ -112,12 +144,93 @@ The solution to this is to also prefix every imported name with ``_``: import asyncio as _asyncio import tabnanny as _tabnanny -This brings us back to the original problem: this sprinkles the -code with extra underscores, and puts mental overhead on the developer -by requiring them to remember to prefix their imports with ``_``. -Ideally, users shouldn't be tempted to reach for private names from modules in -the first place. + +.. _pep-842-prefixed-public: + +Prefixed names are not a universal rule +*************************************** + +As modules evolve, some underscored names are made public, either because users +did not clearly understand that an underscore indicated instability, or because +users found useful functionality in a module's private API, and nothing was +discouraging them from using it. + +In the standard library, a prime example of this is the :mod:`ctypes` module. +``ctypes`` is full of public APIs that are subject to Python's backwards +compatibility policy, but contain a leading underscore. For example: + +1. :class:`ctypes._CFuncPtr` +2. :class:`ctypes._CData` +3. :class:`ctypes._Pointer` + +This sends the wrong message to consumers of the API. When seeing things like +this in a codebase, it makes it seem like the code is opting out of backwards +compatibility, or that an underscored name does not mean "private" in the +module. In both cases, consumers are inclined to reach for more private names, +making this problem worse. + + +We want to be nice to users, not shrug them away +------------------------------------------------ + +When a user decides to use a private API, accidentally or not, they will +inevitably be broken by the library author(s). In many cases, this results +in a bug report asking for the API to be fixed or restored to prevent +downstream breakage. In this case, the library maintainer has to make a decision: + +1. Tell the user that they're in the wrong for using it, and allow the breakage + to take place. +2. Commit to maintaining the private API as public, increasing the burden on + themselves and encountering some of the problems described in + :ref:`pep-842-prefixed-public`. + +This PEP is not intended to solve this problem entirely, but instead is meant +to mitigate it by making it much clearer that a user is accessing a private name; +in other words, this PEP wants to decrease (or eliminate) the amount of accidental +private API usage in practice. By accessing a private API, the user must make a +conscious decision to do so. + + +Library consumers use runtime introspection for documentation +************************************************************* + +A counterargument to the above section is that a library should clearly +document what is private and what is public. In theory, yes, but in practice, +users don't read the documentation in full. + +A common practice when designing APIs is to design for intuition. If an API +is named and placed well, then a user often won't need to reach for the +documentation. Python is no exception to this. + +When prototyping, it's typical for someone to use :func:`dir` or :func:`help` +in Python's interactive :term:`REPL` to look for attributes or methods that +are useful to them. In this case, if something is intuitive enough for the +user, they will simply reach for it without checking the documentation first. +In a language as dynamic as Python, the way people consume APIs is also dynamic. + + +``__all__`` is only a convention +-------------------------------- + +The fundamental issue here is that Python has no way to express which names +in a module are "private" or "public" -- or, in other words, which names are +designed to be stable APIs for users. Prefixing is an option, but given the +reasons above, it's not always a bulletproof (or nice) solution for library +authors. + +Currently, the convention for expressing which names are public is done through +a module's :attr:`__all__` variable. This has two major downsides: + +1. ``__all__`` often gets out of sync, because as developers add, change, or + remove names from their module, there is often nothing pushing them towards + changing ``__all__``, because again, using it to list public names is only + a *convention* and not enforced by anything. +2. ``__all__`` is not always exhaustive. See the :ref:`rejected ideas + ` for examples on where the items in ``__all__`` + might only be a subset of the "public" names in a module. + +This PEP intends to solve both of these problems with a new ``__export__`` variable. Specification @@ -129,6 +242,7 @@ Specification ``__export__`` rules -------------------- + Object requirements ******************* @@ -180,8 +294,8 @@ Module attribute access When ``__export__`` is present in a module's globals, all access to attributes present on the module object will also check if the attribute name is present in ``__export__`` (via ``__contains__`` or through iteration, as specified previously). -If the attribute name is not present in ``__export__``, then an :exc:`ImportError` -is raised. For example: +If the attribute name is not present in ``__export__``, then an :exc:`RuntimeWarning` +is emitted. For example: .. code-block:: python @@ -197,10 +311,14 @@ is raised. For example: >>> spam.a 42 >>> spam.b - Traceback (most recent call last): - File "", line 1, in - spam.b - ImportError: 'b' is not exported by 'spam' + :1: RuntimeWarning: 'b' is not exported by 'spam' + 24 + + +.. note:: + + This also affects ``from`` imports, because that uses the same attribute + access mechanism. Dunder names @@ -223,32 +341,6 @@ For example: 'spam' -Lazy imports -************ - -:ref:`Lazy imports ` that are not listed in ``__export__`` -will not be reified upon being accessed outside the module. For example: - -.. code-block:: python - - # spam.py - lazy import json - - __export__ = [] - -.. code-block:: pycon - - >>> import spam, sys - >>> assert 'json' in sys.lazy_modules - >>> spam.json - Traceback (most recent call last): - File "", line 1, in - spam.json - ImportError: 'json' is not exported by 'spam' - >>> # json is still lazy and has not been resolved - >>> assert 'json' in sys.lazy_modules - - Module ``__getattr__`` functions -------------------------------- @@ -337,16 +429,6 @@ names that are not present in ``__export__``. For example: >>> dir(spam) [..., 'a', 'b'] -It is worth noting that there are real consequences for including unexported -names in custom ``__dir__`` functions. For example, :func:`help` can no longer -be used with the above module: - -.. code-block:: pycon - - >>> import spam - >>> help(spam) - 'b' is not exported by 'spam' - .. _pep-842-implicit-all: @@ -425,7 +507,7 @@ following code: return value if name not in __export__: - raise ImportError(f"{name!r} is not exported by {__name__!r}") + __import__("warnings").warn(f"{name!r} is not exported by {__name__!r}", RuntimeWarning, stacklevel=1) return value @@ -444,14 +526,15 @@ following code: Rationale ========= +.. _pep-842-not-an-access-modifier: -``__export__`` is not a secure access modifier ----------------------------------------------- +``__export__`` is not an access modifier +---------------------------------------- -This PEP does not aim to be a secure mechanism for preventing access to -private attributes in modules. In fact, bypassing ``__export__`` is trivial; -simply access ``mod.__dict__['attr_name']`` instead of ``mod.attr_name`` at -runtime. +This PEP does not aim to be a mechanism for preventing access to private +attributes in modules. The :exc:`RuntimeWarning` can be filtered away, +disabled, or bypassed (such as by accessing attributes through the module's +``__dict__``). This is by design. Python does not include access modifiers as a language feature for a reason. To `quote `__ Eric Smith: @@ -500,11 +583,21 @@ Reference Implementation A reference implementation of this PEP can be found `here `__. +Performance +*********** + +The reference implementation does not currently implement any optimizations +to reduce the overhead of the ``__export__`` lookup or iteration, meaning +that there is likely some overhead. However, if this PEP is accepted, +optimizations will be implemented before the feature lands in :term:`CPython`. + Rejected Ideas ============== +.. _pep-842-all-for-exports: + Reuse ``__all__`` for exports ----------------------------- @@ -549,6 +642,29 @@ syntax. Third-party solutions and widespread adoption would make it much clearer that new syntax is the best choice for Python in the long run. +Raising an exception upon accessing unexported attributes +--------------------------------------------------------- + +This PEP initially proposed raising an :exc:`ImportError` upon accessing +module attributes that were not listed in ``__export__``. For example: + +.. code-block:: pycon + + >>> import module + >>> module.unexported + Traceback (most recent call last): + File "", line 1, in + module.unexported + ImportError: 'unexported' is not exported by 'module' + + +This caused a lot of concern, as many were fundamentally uncomfortable with +the idea of introducing any notion of "private attributes" in Python. The purpose +of this proposal is to improve *expression* of private variables, not *security*. +As such, this proposal switched to emitting warnings when accessing unexported +names. + + Open Issues =========== @@ -566,7 +682,10 @@ behind this PEP. Change History ============== -TBD. +- 01-Aug-2026: + * Accessing an unexported attribute now emits a :exc:`RuntimeWarning` instead + of raising an :exc:`ImportError`. + * Significantly overhauled the motivation section. Copyright From d751a479c6a80d0ed8b8dde470674f0439fb69e7 Mon Sep 17 00:00:00 2001 From: Peter Bierma Date: Sat, 1 Aug 2026 11:06:23 -0400 Subject: [PATCH 2/4] Fix build errors. --- peps/pep-0842.rst | 54 ++++++++++++++++++++++++++--------------------- 1 file changed, 30 insertions(+), 24 deletions(-) diff --git a/peps/pep-0842.rst b/peps/pep-0842.rst index 02f453918d2..94e72ff1043 100644 --- a/peps/pep-0842.rst +++ b/peps/pep-0842.rst @@ -14,7 +14,7 @@ Abstract ======== This PEP proposes an ``__export__`` variable that modules can define to -express intent behind visibility to variables from outside the module. +express intent about the visibility of variables from outside the module. For example: @@ -52,8 +52,8 @@ Motivation ========== -Module names need privacy -------------------------- +Module-level names need privacy +------------------------------- A developer is writing a Python module. The module is intended to have one "public" class -- a class that is intended for users of the module -- called @@ -105,7 +105,7 @@ convention; even if this PEP is accepted, it's expected that "underscored" names (names prefixed with a leading ``_``) will remain a staple of Python for years to come. The purpose of this PEP is not to eliminate the need for ``_`` in module-level names, but instead to clear up corner cases where a -private name is ambigious or tempting. In other words, this PEP is intended +private name is ambiguous or tempting. In other words, this PEP is intended to improve expressiveness and clarity with private APIs, *not* to add brand new functionality. @@ -145,6 +145,8 @@ The solution to this is to also prefix every imported name with ``_``: import tabnanny as _tabnanny +But, again, this sprinkles the code with even more underscored names. + .. _pep-842-prefixed-public: @@ -167,15 +169,15 @@ compatibility policy, but contain a leading underscore. For example: This sends the wrong message to consumers of the API. When seeing things like this in a codebase, it makes it seem like the code is opting out of backwards compatibility, or that an underscored name does not mean "private" in the -module. In both cases, consumers are inclined to reach for more private names, -making this problem worse. +module. In both cases, consumers are inclined to reach for more private names +(because there's no apparent consequence for doing so), making this problem worse. We want to be nice to users, not shrug them away ------------------------------------------------ When a user decides to use a private API, accidentally or not, they will -inevitably be broken by the library author(s). In many cases, this results +inevitably be broken by the library author. In many cases, this results in a bug report asking for the API to be fixed or restored to prevent downstream breakage. In this case, the library maintainer has to make a decision: @@ -204,31 +206,32 @@ is named and placed well, then a user often won't need to reach for the documentation. Python is no exception to this. When prototyping, it's typical for someone to use :func:`dir` or :func:`help` -in Python's interactive :term:`REPL` to look for attributes or methods that -are useful to them. In this case, if something is intuitive enough for the -user, they will simply reach for it without checking the documentation first. -In a language as dynamic as Python, the way people consume APIs is also dynamic. +in Python's interactive :term:`REPL` to look for attributes that are useful to +them. In this case, if something is intuitive enough for the user, they will +simply reach for it without checking the documentation first. In a language +as dynamic as Python, the way people consume APIs is also dynamic. ``__all__`` is only a convention -------------------------------- The fundamental issue here is that Python has no way to express which names -in a module are "private" or "public" -- or, in other words, which names are -designed to be stable APIs for users. Prefixing is an option, but given the +in a module are "private" or "public". Prefixing is an option, but given the reasons above, it's not always a bulletproof (or nice) solution for library authors. -Currently, the convention for expressing which names are public is done through -a module's :attr:`__all__` variable. This has two major downsides: +Currently, the other convention for expressing which names are public is +done through a module's ``__all__`` variable. This has two major downsides: 1. ``__all__`` often gets out of sync, because as developers add, change, or remove names from their module, there is often nothing pushing them towards changing ``__all__``, because again, using it to list public names is only - a *convention* and not enforced by anything. + a convention and not enforced by anything. 2. ``__all__`` is not always exhaustive. See the :ref:`rejected ideas ` for examples on where the items in ``__all__`` - might only be a subset of the "public" names in a module. + might only be a subset of the "public" names in a module. In short, it can + be difficult to control namespace pollution and declare all public names in + ``__all__`` simultaneously. This PEP intends to solve both of these problems with a new ``__export__`` variable. @@ -294,7 +297,7 @@ Module attribute access When ``__export__`` is present in a module's globals, all access to attributes present on the module object will also check if the attribute name is present in ``__export__`` (via ``__contains__`` or through iteration, as specified previously). -If the attribute name is not present in ``__export__``, then an :exc:`RuntimeWarning` +If the attribute name is not present in ``__export__``, then a :exc:`RuntimeWarning` is emitted. For example: .. code-block:: python @@ -311,7 +314,7 @@ is emitted. For example: >>> spam.a 42 >>> spam.b - :1: RuntimeWarning: 'b' is not exported by 'spam' + :1: RuntimeWarning: 'b' is not exported by 'spam' 24 @@ -492,11 +495,13 @@ following code: .. code-block:: python - __all__ = __export__ + if "__all__" not in globals(): + __all__ = __export__ def _is_dunder_name(name): return (len(name) > 4) and name.startswith("__") and name.endswith("__") + # Attributes not in the __dict__ fall back to the normal lookup def __getattribute__(name): try: value = globals()[name] @@ -584,7 +589,7 @@ A reference implementation of this PEP can be found `here `__. Performance -*********** +----------- The reference implementation does not currently implement any optimizations to reduce the overhead of the ``__export__`` lookup or iteration, meaning @@ -682,10 +687,11 @@ behind this PEP. Change History ============== -- 01-Aug-2026: - * Accessing an unexported attribute now emits a :exc:`RuntimeWarning` instead +* 01-Aug-2026 + + - Accessing an unexported attribute now emits a :exc:`RuntimeWarning` instead of raising an :exc:`ImportError`. - * Significantly overhauled the motivation section. + - Significantly overhauled the motivation section. Copyright From 14583c8e5b86bfee2d0dff588ff3bf8e13a8d8fc Mon Sep 17 00:00:00 2001 From: Peter Bierma Date: Sat, 1 Aug 2026 11:12:03 -0400 Subject: [PATCH 3/4] Fix typo. --- peps/pep-0842.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/peps/pep-0842.rst b/peps/pep-0842.rst index 94e72ff1043..d43d453f49d 100644 --- a/peps/pep-0842.rst +++ b/peps/pep-0842.rst @@ -320,7 +320,7 @@ is emitted. For example: .. note:: - This also affects ``from`` imports, because that uses the same attribute + This also affects ``from`` imports, because those use the same attribute access mechanism. From 32c8bcc48dbccfa21b0dd6409718dbf8d4bcd3df Mon Sep 17 00:00:00 2001 From: Peter Bierma Date: Sat, 1 Aug 2026 11:14:13 -0400 Subject: [PATCH 4/4] One more minor change. --- peps/pep-0842.rst | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/peps/pep-0842.rst b/peps/pep-0842.rst index d43d453f49d..4396efa8519 100644 --- a/peps/pep-0842.rst +++ b/peps/pep-0842.rst @@ -217,8 +217,7 @@ as dynamic as Python, the way people consume APIs is also dynamic. The fundamental issue here is that Python has no way to express which names in a module are "private" or "public". Prefixing is an option, but given the -reasons above, it's not always a bulletproof (or nice) solution for library -authors. +reasons above, it's not always a bulletproof solution for library authors. Currently, the other convention for expressing which names are public is done through a module's ``__all__`` variable. This has two major downsides: