diff --git a/peps/pep-0843.rst b/peps/pep-0843.rst index 49b35bbf4d9..741f7e15be8 100644 --- a/peps/pep-0843.rst +++ b/peps/pep-0843.rst @@ -14,21 +14,22 @@ Post-History: `05-Aug-2026 `__, Abstract ======== -Large libraries separate their **implementation layout** (the tree of -modules convenient for maintainers) from their **public layout** (the -shallower, curated tree they present to users). Building that public layout -today means choosing between two imperfect options. - -The first is writing every exported name twice: once in an import -statement, again as a string in ``__all__``. The two lists must be kept -in sync by hand every time the public layout changes. - -The second is the reflexive-alias idiom, ``from x import y as y``. This -is part of the type system: type checkers treat it as a signal that the -import is an intentional re-export. It still reads like a typo to anyone -who doesn't know the convention. Because the module has no curated -``__all__``, it loses the wildcard-import control a real ``__all__`` -gives. +Large libraries separate their **implementation layout** (the tree of modules +convenient for maintainers) from their **public layout** (the shallower, +curated tree they present to users). Building that public layout today means +choosing between two imperfect options. + +The first is writing every exported name twice: once in an import statement, +again as a string in ``__all__``. The two lists must be kept in sync by hand +every time the public layout changes, violating *DRY* (Don't Repeat Yourself): +"Every piece of knowledge must have a single, unambiguous, authoritative +representation within a system." [#pragprog]_ + +The second is the reflexive-alias idiom, ``from x import y as y``. This is part +of the type system: type checkers treat it as a signal that the import is an +intentional re-export. It still reads like a typo to anyone who doesn't know +the convention. Because the module has no curated ``__all__``, it loses the +wildcard-import control a real ``__all__`` gives. This PEP adds a statement form that avoids both problems: @@ -38,19 +39,19 @@ This PEP adds a statement form that avoids both problems: from ._internal.core export PublicAPI from ._internal.widgets export Widget as PublicWidget -It imports the name, optionally under an alias exactly as ``from ... -import ... as ...`` does, and appends it to ``__all__`` in the same -statement. Nothing is left to sync by hand, and no alias needs decoding. +It imports the name, optionally under an alias exactly as ``from ... import ... +as ...`` does, and appends it to ``__all__`` in the same statement. Nothing is +left to sync by hand, and no alias needs decoding. Relationship to PEP 842 ======================= -Both PEPs start from the same discomfort with ``__all__``, and agree on -the same core mechanism for re-exports: a statement of the shape ``from - export ``, with a lazy variant (see `Lazy exports`_). -That agreement, reached independently, is confirmation that this is the -right shape for re-exports. +Both PEPs start from the same discomfort with ``__all__``, and agree on the +same core mechanism for re-exports: a statement of the shape ``from +export ``, with a lazy variant (see `Lazy exports`_). That agreement, +reached independently, is confirmation that this is the right shape for +re-exports. PEP 842 broadens the mechanism into a keyword usable in five forms: @@ -60,12 +61,12 @@ PEP 842 broadens the mechanism into a keyword usable in five forms: * ``export class`` * A module re-export statement, ``from MODULE export NAME`` -All five populate ``__export__``, which also becomes ``__all__`` and -triggers an ``ExportError`` on access to anything left out. +All five populate ``__export__``, which also becomes ``__all__`` and triggers +an ``ExportError`` on access to anything left out. -PEP 842's version has no wildcard equivalent to `Wildcard form`_. This -PEP takes only the module re-export statement, deliberately leaving out -the rest; see `Non-goals`_ for what and why. +PEP 842's version has no wildcard equivalent to `Wildcard form`_. This PEP +takes only the module re-export statement, deliberately leaving out the rest; +see `Non-goals`_ for what and why. Motivation @@ -101,53 +102,50 @@ this: # ... the same names again ... ] -This is the file where a library flattens its implementation layout into -its public layout. As a library grows, the two diverge: code gets -reorganized into submodules for the maintainer's convenience, while the -public layout stays stable for users. +This is the file where a library flattens its implementation layout into its +public layout. As a library grows, the two diverge: code gets reorganized into +submodules for the maintainer's convenience, while the public layout stays +stable for users. -Something has to do the flattening. Today that something is a -hand-maintained, doubly-written list: the export list (in ``__all__``) -and the import list (of ``import`` statements) say the same thing -twice. Any rename, addition, or removal has to be made in two places by -hand, and the two can silently drift apart. That's the DRY violation -this PEP removes by folding both into one ``from export -`` statement. +Something has to do the flattening. Today that something is a hand-maintained, +doubly-written list: the export list (in ``__all__``) and the import list (of +``import`` statements) say the same thing twice. Any rename, addition, or +removal has to be made in two places by hand, and the two can silently drift +apart. This PEP removes that duplication by folding both into one ``from + export `` statement. -The alternative, ``import x as x``, is a workaround for the language's -missing export concept, and it still trips up some auto-formatters, which -see a bare ``import x`` as unused and remove it. +The alternative, ``import x as x``, is a workaround for the language's missing +export concept, and it still trips up some auto-formatters, which see a bare +``import x`` as unused and remove it. ``__all__`` conflates two concerns ---------------------------------- -Hand-maintained ``__all__`` also mixes two distinct concerns in one -file: the list of imports (an implementation detail of how the -flattening is wired up) and the declaration of the public API (a promise -to users). The two live in separate statements at different places in -the file, and nothing keeps them in sync except a reviewer checking them -against each other by eye, or a linter rule built for exactly this case. -The more common "unused import" checks don't help: if a name is -imported but never added to ``__all__``, that name reads as unused, and -autofixers routinely delete it rather than surface the omission. +Hand-maintained ``__all__`` also mixes two distinct concerns in one file: the +list of imports (an implementation detail of how the flattening is wired up) +and the declaration of the public API (a promise to users). The two live in +separate statements at different places in the file, and nothing keeps them in +sync except a reviewer checking them against each other by eye, or a linter +rule built for exactly this case. The more common "unused import" checks don't +help: if a name is imported but never added to ``__all__``, that name reads as +unused, and autofixers routinely delete it rather than surface the omission. Underscores solve a different problem ------------------------------------- -A natural response is: "just prefix internal names with an underscore." -But the privacy this PEP cares about lives at the package level, not the -name level: which parts of a large, multi-module package belong in the -**public layout**. Underscore-prefixing already marks a name private -within a module. +A natural response is: "just prefix internal names with an underscore." But the +privacy this PEP cares about lives at the package level, not the name level: +which parts of a large, multi-module package belong in the **public layout**. +Underscore-prefixing already marks a name private within a module. -The problem shows up in "hub" modules (usually ``__init__.py`` files) -whose only job is gathering names from internal modules and presenting -them under a stable public name. Every name that reaches a hub module is -already meant to be public: the underscore convention has done its job -by the time the name enters the hub. Hub modules need a non-repetitive -way to say "this is also part of the package's public layout." +The problem shows up in "hub" modules (usually ``__init__.py`` files) whose +only job is gathering names from internal modules and presenting them under a +stable public name. Every name that reaches a hub module is already meant to be +public: the underscore convention has done its job by the time the name enters +the hub. Hub modules need a non-repetitive way to say "this is also part of the +package's public layout." Non-goals @@ -157,9 +155,9 @@ This PEP does not aim to: * Restrict runtime attribute access to non-exported names, or change ``__getattr__`` semantics. See `Why no runtime enforcement`_. -* Mark a freshly written ``def``, ``class``, or assignment as exported at - its definition site, the way the third-party ``atpublic`` package does - with ``@public``/``@private`` decorators. See `Why only re-exports`_. +* Mark a freshly written ``def``, ``class``, or assignment as exported at its + definition site, the way the third-party ``atpublic`` package does with + ``@public``/``@private`` decorators. See `Why only re-exports`_. Specification @@ -175,10 +173,20 @@ statement: from numpy.typing export NDArray A ``from export [as ]`` statement does what ``from - import [as ]`` does: it binds ````, or -```` if given, in the current namespace, and also appends that name -to the module's ``__all__``, creating ``__all__`` if it doesn't already -exist. + import [as ]`` does: it binds ````, or ```` +if given, in the current namespace, and appends that name to ``__all__``: + +.. code-block:: python + + from import as + exported_names = globals().setdefault("__all__", []) + if not isinstance(exported_names, list): + exported_names = list(exported_names) + __all__ = exported_names + exported_names.append("") + +Every other statement form in this PEP normalizes ``__all__`` the same way +before appending or extending it. Because it desugars to an ordinary import plus an append to ``__all__``, ``export`` composes with control flow exactly as ``import`` does: @@ -190,18 +198,22 @@ Because it desugars to an ordinary import plus an append to ``__all__``, else: from ._internal.posix export PosixThing -Each branch runs its own import and its own ``__all__`` append, so the -name that ends up exported depends on which branch ran, with no separate -``__all__`` bookkeeping required. - -More generally, ``export`` is valid everywhere ``import`` is valid: -inside functions, classes, ``try`` blocks, and anywhere else a statement -can appear, with no restriction of its own. If ``export`` should be -limited in some context, ``import`` would need the same limit; that's -the subject of a separate proposal, not this one. - -In particular, using ``export`` in an ``if typing.TYPE_CHECKING:`` guard -lets stub-only packages such as ``_typeshed`` export a name that exists +Each branch runs its own import and its own ``__all__`` append, so the name +that ends up exported depends on which branch ran, with no separate ``__all__`` +bookkeeping required. + +Unlike ``import``, ``export`` is restricted to module level: it's a +``SyntaxError`` inside a ``def`` or ``class`` body, though it may still appear +inside ``if``, ``try``, ``for``, ``while``, or ``with`` blocks, as in the +platform example above, since those don't introduce a new scope. The +restriction exists because a name bound inside a function or class body was +never part of the module's namespace to begin with, so there's nothing there +for ``export`` to add to ``__all__``: the whole point of ``export`` is +populating the *module's* public API, and only names bound at module level +qualify. + +In particular, using ``export`` in a module-level ``if typing.TYPE_CHECKING:`` +guard lets stub-only packages such as ``_typeshed`` export a name that exists in the stub but has no runtime counterpart. .. code-block:: python @@ -209,12 +221,11 @@ in the stub but has no runtime counterpart. if typing.TYPE_CHECKING: from ._internal.types export InternalOnly -```` may be relative (``from .core export Thing``, ``from -..sub.core export Thing``) or absolute (``from numpy.typing export -NDArray``), exactly as in an ordinary ``from ... import ...`` statement. -A single statement may export multiple names, using the same syntax as a -regular multi-name ``from`` import, including a parenthesized, multi-line -list for long ones: +```` may be relative (``from .core export Thing``, ``from ..sub.core +export Thing``) or absolute (``from numpy.typing export NDArray``), exactly as +in an ordinary ``from ... import ...`` statement. A single statement may export +multiple names, using the same syntax as a regular multi-name ``from`` import, +including a parenthesized, multi-line list for long ones: .. code-block:: python @@ -227,21 +238,20 @@ list for long ones: ) Exporting a name is itself a use of it, so tools that flag "imported but -unused" names (linters, formatters) should treat every name bound by a -``from export ...`` statement as used, the same way they already -special-case ``from module import Thing as Thing``. This PEP doesn't -change what those tools decide. It follows from what ``export`` means: the -export is the use. +unused" names (linters, formatters) should treat every name bound by a ``from + export ...`` statement as used, the same way they already special-case +``from module import Thing as Thing``. This PEP doesn't change what those tools +decide. It follows from what ``export`` means: the export is the use. Wildcard form ------------- -This proposal also includes a wildcard form, ``from export *``. -It binds every name that ``from import *`` would bind, using the -same rule (````'s own ``__all__`` if it defines one, otherwise -every top-level name that doesn't start with an underscore), and appends -all of those names to the current module's ``__all__``: +This proposal also includes a wildcard form, ``from export *``. It +binds every name that ``from import *`` would bind, using the same +rule (````'s own ``__all__`` if it defines one, otherwise every +top-level name that doesn't start with an underscore), and appends all of those +names to the current module's ``__all__``: .. code-block:: python @@ -257,30 +267,31 @@ This supports a common two-tier layout: an internal module curates its own ``__all__`` as it's written, and the hub re-exports that whole list in one statement, instead of naming each item again. -``export *`` matches ``import *``'s fallback when ```` defines -no ``__all__`` of its own. It exports every top-level name that doesn't -start with an underscore. +``export *`` matches ``import *``'s fallback when ```` defines no +``__all__`` of its own. It exports every top-level name that doesn't start +with an underscore. -The wildcard form is equivalent to: +The wildcard form is equivalent to, normalizing ``__all__`` as in +`Specification`_: .. code-block:: python # from ._internal.core export * from ._internal.core import * - __all__ = list(globals().get("__all__", [])) + _names_bound_by_star_import + __all__.extend(_names_bound_by_star_import) where ``_names_bound_by_star_import`` is the list of names ``from -._internal.core import *`` just bound, the same list Python's import -machinery already computes to execute a wildcard import. +._internal.core import *`` just bound, the same list Python's import machinery +already computes to execute a wildcard import. Lazy exports ------------ :pep:`810` adds a ``lazy`` soft keyword that defers a ``from ... import`` -statement until the imported name is first used: ``lazy from -import `` binds a lazy proxy immediately but doesn't load -```` until that proxy is touched. +statement until the imported name is first used: ``lazy from import +`` binds a lazy proxy immediately but doesn't load ```` until +that proxy is touched. ``export`` composes with it the same way it composes with ``import``: @@ -288,34 +299,33 @@ import `` binds a lazy proxy immediately but doesn't load lazy from ._internal.core export PublicAPI -This binds ``PublicAPI`` to a lazy proxy, exactly as :pep:`810` specifies, -and appends ``"PublicAPI"`` to ``__all__`` immediately, without waiting -for the proxy to be touched. Populating ``__all__`` only needs the name as -a string, not the loaded value, so the export half of the statement stays -eager even when the import half is lazy. For a hub module with hundreds of -re-exports, this gives users a complete, accurate ``__all__`` and -``dir()`` at import time, without paying the cost of loading every -internal module up front. +This binds ``PublicAPI`` to a lazy proxy, exactly as :pep:`810` specifies, and +appends ``"PublicAPI"`` to ``__all__`` immediately, without waiting for the +proxy to be touched. Populating ``__all__`` only needs the name as a string, +not the loaded value, so the export half of the statement stays eager even when +the import half is lazy. For a hub module with hundreds of re-exports, this +gives users a complete, accurate ``__all__`` and ``dir()`` at import time, +without paying the cost of loading every internal module up front. -The statement is equivalent to: +The statement is equivalent to, normalizing ``__all__`` as in `Specification`_: .. code-block:: python # lazy from ._internal.core export PublicAPI lazy from ._internal.core import PublicAPI - __all__ = list(globals().get("__all__", [])) + ["PublicAPI"] + __all__.append("PublicAPI") -``lazy from export *`` is not allowed, for two independent -reasons: :pep:`810` already disallows ``lazy from import *``, and -the wildcard export form needs ```` loaded to know what names -``__all__`` even contains, which is exactly what laziness defers. ``lazy`` -also inherits :pep:`810`'s scope restriction: it's only valid at module -level, not inside functions, classes, or ``try`` blocks. +``lazy from export *`` is not allowed, for two independent reasons: +:pep:`810` already disallows ``lazy from import *``, and the wildcard +export form needs ```` loaded to know what names ``__all__`` even +contains, which is exactly what laziness defers. ``lazy`` also inherits +:pep:`810`'s scope restriction: it's only valid at module level, not inside +functions, classes, or ``try`` blocks. -NumPy's ``numpy/__init__.py`` illustrates why this matters. Its -module-level ``__getattr__`` does two unrelated jobs at once: lazily -loading submodules that aren't imported at ``import numpy`` time, and -raising helpful errors for attributes that no longer exist: +NumPy's ``numpy/__init__.py`` illustrates why this matters. Its module-level +``__getattr__`` does two unrelated jobs at once: lazily loading submodules that +aren't imported at ``import numpy`` time, and raising helpful errors for +attributes that no longer exist: .. code-block:: python @@ -335,8 +345,7 @@ raising helpful errors for attributes that no longer exist: raise AttributeError(f"`np.{attr}` was removed. ...") raise AttributeError(f"module {__name__!r} has no attribute {attr!r}") -Only the first job is a laziness concern, and lazy exports replace it -directly: +Only the first job is a laziness concern, and lazy exports replace it directly: .. code-block:: python @@ -352,46 +361,43 @@ directly: raise AttributeError(f"module {__name__!r} has no attribute {attr!r}") This isn't only shorter, it's more correct. Today, ``linalg`` exists only -through the ``__getattr__`` fallback, so it's invisible to ``dir(numpy)`` -and tab completion unless NumPy separately maintains a ``__dir__`` -override listing it. ``lazy from . export linalg`` binds a real (lazy) -attribute immediately and adds ``"linalg"`` to ``__all__``, so ``dir()`` -and ``__all__`` are correct automatically, and ``__getattr__`` is no -longer even called for these names, since ordinary attribute lookup now -succeeds before it would run. +through the ``__getattr__`` fallback, so it's invisible to ``dir(numpy)`` and +tab completion unless NumPy separately maintains a ``__dir__`` override listing +it. ``lazy from . export linalg`` binds a real (lazy) attribute immediately and +adds ``"linalg"`` to ``__all__``, so ``dir()`` and ``__all__`` are correct +automatically, and ``__getattr__`` is no longer even called for these names, +since ordinary attribute lookup now succeeds before it would run. The second job, warning for attributes that no longer exist at all, isn't -something ``export`` addresses. ``export`` only concerns names that -should be bound; it has nothing to say about names that were removed. A -module ``__getattr__`` is still needed for that, just a smaller one, with -only the deprecation logic left in it once the lazy-submodule branches -move out. +something ``export`` addresses. ``export`` only concerns names that should be +bound; it has nothing to say about names that were removed. A module +``__getattr__`` is still needed for that, just a smaller one, with only the +deprecation logic left in it once the lazy-submodule branches move out. -One restriction matters for this rewrite: :pep:`810` disallows ``lazy`` -inside function bodies, so the lazy-submodule branches must move out of -``__getattr__`` to module top level, not be replaced line by line inside -it. That's a restructuring, not a drop-in substitution, though it's also -exactly the shape a hub module (`How to Teach This`_) already takes. +One restriction matters for this rewrite: :pep:`810` disallows ``lazy`` inside +function bodies, so the lazy-submodule branches must move out of +``__getattr__`` to module top level, not be replaced line by line inside it. +That's a restructuring, not a drop-in substitution, though it's also exactly +the shape a hub module (`How to Teach This`_) already takes. Interaction with ``__all__`` ---------------------------- -A module may freely mix ``from ... export ...`` statements with a -manually maintained ``__all__``, or with ``__all__ +=`` / -``__all__.append`` calls elsewhere in the file. Each ``export`` -statement looks at whatever is currently bound to ``__all__`` in the -module's namespace before appending the new name(s): +A module may freely mix ``from ... export ...`` statements with a manually +maintained ``__all__``, or with ``__all__ +=`` / ``__all__.append`` calls +elsewhere in the file. Each ``export`` statement looks at whatever is currently +bound to ``__all__`` in the module's namespace before appending the new +name(s): -* If ``__all__`` doesn't exist yet, ``export`` creates it, as an empty - list. -* If ``__all__`` exists but isn't already a list, ``export`` copies it - into a list, preserving its existing contents. +* If ``__all__`` doesn't exist yet, ``export`` creates it, as an empty list. +* If ``__all__`` exists but isn't already a list, ``export`` copies it into a + list, preserving its existing contents. * Otherwise ``__all__`` is already a list, and is used as is. The new name is then appended. So an ``export`` statement always leaves -``__all__`` as an ordinary, mutable list, one that later code in the -same module can keep extending with plain list operations: +``__all__`` as an ordinary, mutable list, one that later code in the same +module can keep extending with plain list operations: .. code-block:: python @@ -402,45 +408,39 @@ same module can keep extending with plain list operations: Duplicate names are allowed: ``__all__`` was never required to be free of duplicates, and this PEP doesn't change that. -``from ... export ...`` affects only the contents of ``__all__``, which in -turn affects ``from module import *`` and any tool that already reads -``__all__`` (documentation generators, linters, IDEs). +``from ... export ...`` affects only the contents of ``__all__``, which in turn +affects ``from module import *`` and any tool that already reads ``__all__`` +(documentation generators, linters, IDEs). -``export`` cannot hide the intermediate submodule(s) named in ```` -from a hub module's own ``dir()``. When ``spam/__init__.py`` contains -``from ._internal.core export PublicAPI``, ``_internal`` becomes an -attribute of the ``spam`` module and shows up in ``dir(spam)``, regardless -of whether the statement uses ``import``, ``from ... import``, or -``export``. ``spam/__init__.py``'s own namespace *is* ``spam.__dict__``, -and Python's import system binds an imported submodule onto its parent -package's namespace as a side effect of importing it, for both packages -and plain modules. This is a property of the import system, not something -``export`` introduces or can suppress. It is one more reason -attribute-access hiding is a `non-goal `_ of this PEP. +``export`` cannot hide the intermediate submodule(s) named in ```` from +a hub module's own ``dir()``. When ``spam/__init__.py`` contains ``from +._internal.core export PublicAPI``, ``_internal`` becomes an attribute of the +``spam`` module and shows up in ``dir(spam)``, regardless of whether the +statement uses ``import``, ``from ... import``, or ``export``. +``spam/__init__.py``'s own namespace *is* ``spam.__dict__``, and Python's +import system binds an imported submodule onto its parent package's namespace +as a side effect of importing it, for both packages and plain modules. This is +a property of the import system, not something ``export`` introduces or can +suppress. It is one more reason attribute-access hiding is a `non-goal +`_ of this PEP. Semantic implementation ----------------------- -Each ``from export as `` statement is equivalent -to: - -.. code-block:: python - - from import as - __all__ = list(globals().get("__all__", [])) + [""] - +Each ``from export as `` statement desugars exactly as +shown in `Specification`_: import the name, normalize ``__all__``, then append. For example: .. code-block:: python # from ._internal.core export PublicAPI from ._internal.core import PublicAPI - __all__ = list(globals().get("__all__", [])) + ["PublicAPI"] + __all__.append("PublicAPI") # from ._internal.widgets export Widget as PublicWidget from ._internal.widgets import Widget as PublicWidget - __all__ = list(globals().get("__all__", [])) + ["PublicWidget"] + __all__.append("PublicWidget") The wildcard form's equivalent is given in `Wildcard form`_, and the lazy form's in `Lazy exports`_. @@ -455,94 +455,89 @@ Rationale Why a keyword and not a decorator --------------------------------- -A ``@public``-style decorator, as in the third-party ``atpublic`` -package, works neatly for individually defined functions and classes, -but it doesn't compose with ``import`` statements: there's no object to -decorate when the "definition" is just a name entering the module -through an import. +A ``@public``-style decorator, as in the third-party ``atpublic`` package, +works neatly for individually defined functions and classes, but it doesn't +compose with ``import`` statements: there's no object to decorate when the +"definition" is just a name entering the module through an import. + ``atpublic`` works around this with a function-call form, -``public(some_imported_name)``, but that reintroduces the double-write -this PEP removes: the name is written once in the import and again as an -argument to ``public()``. It also doesn't compose with aliases: the -function-call form only takes keyword arguments, ``public(alias=name)``, -which both adds ``alias`` to ``__all__`` and binds it, so publishing an -alias means spelling out the mapping in the call instead of using -``from x import y as z``. A statement-level ``export`` keyword avoids -both problems, because it's part of the import statement itself; it -adds nothing beyond the import that would exist anyway. +``public(some_imported_name)``, but that reintroduces the double-write this PEP +removes: the name is written once in the import and again as an argument to +``public()``. It also doesn't compose with aliases: the function-call form only +takes keyword arguments, ``public(alias=name)``, which both adds ``alias`` to +``__all__`` and binds it, so publishing an alias means spelling out the mapping +in the call instead of using ``from x import y as z``. A statement-level +``export`` keyword avoids both problems, because it's part of the import +statement itself; it adds nothing beyond the import that would exist anyway. Why only re-exports ------------------- This PEP deliberately omits a way to mark a fresh ``def``, ``class``, or assignment as exported where it's defined. Some smaller libraries and -single-file modules would rather write ``export def public_function(): -...`` right where the function is defined, but that use case doesn't -share the DRY problem this PEP solves. When a name is defined and -exported in the same place, it's written only once; the maintainer -already chooses whether to write a leading underscore, and tools such as -``atpublic``'s ``@public`` decorator already let that choice happen at -the definition site, without a new statement. - -Smaller libraries and single-file modules that don't organize their -public layout around a re-export hub don't need this PEP at all: -``atpublic`` already serves -them. Conversely, a new project that does adopt hub-and-internals from -the start has little use for ``atpublic`` either: everything meant to be -public is already flowing through the hub's ``export`` statements. If -``atpublic`` gains wide enough adoption regardless, it, or something -like it, may eventually belong in the standard library, independent of -this proposal. +single-file modules would rather write ``export def public_function(): ...`` +right where the function is defined, but that use case doesn't share the DRY +problem this PEP solves. When a name is defined and exported in the same place, +it's written only once; the maintainer already chooses whether to write a +leading underscore, and tools such as ``atpublic``'s ``@public`` decorator +already let that choice happen at the definition site, without a new statement. + +Smaller libraries and single-file modules that don't organize their public +layout around a re-export hub don't need this PEP at all: ``atpublic`` already +serves them. Conversely, a new project that does adopt hub-and-internals from +the start has little use for ``atpublic`` either: everything meant to be public +is already flowing through the hub's ``export`` statements. If ``atpublic`` +gains wide enough adoption regardless, it, or something like it, may eventually +belong in the standard library, independent of this proposal. The evidence gathered for this PEP (NumPy, pandas, polars, Typer, FastAPI, -Plotly) is uniformly about re-export hubs, not about individually defined -names wanting a decorator. A single statement form, one that extends the -familiar ``from ... import ...`` rather than teaching new prefix rules for -``def``, ``class``, and assignment statements, keeps the grammar easy to -describe and easy to review. A later, separate PEP remains free to -propose a definition-site marker if real-world evidence for that gap -emerges; this PEP doesn't need to solve it to solve the re-export problem. +Plotly) is uniformly about re-export hubs, not about individually defined names +wanting a decorator. A single statement form, one that extends the familiar +``from ... import ...`` rather than teaching new prefix rules for ``def``, +``class``, and assignment statements, keeps the grammar easy to describe and +easy to review. A later, separate PEP remains free to propose a definition-site +marker if real-world evidence for that gap emerges; this PEP doesn't need to +solve it to solve the re-export problem. Why no runtime enforcement -------------------------- -The author finds runtime access restriction appealing on its own merits, -and excludes it here purely on scope grounds. :pep:`842` proposes exactly -this: an ``ExportWarning`` when code accesses a non-exported attribute. -Its discussion thread spent considerable effort on whether that access should -warn, raise, or do nothing, and on how such enforcement should interact -with legitimate internal access, drawing substantial pushback over -adversarial framing, per-access performance overhead, unreliable warning -filters, and breakage of patterns like pip's: pip has no public API at -all, yet still supports tools such as pip-tools that deliberately import -``pip._internal``. None of that debate touches the DRY problem this PEP -solves. +The author finds runtime access restriction appealing on its own merits, and +excludes it here purely on scope grounds. :pep:`842` proposes exactly this: an +``ExportWarning`` when code accesses a non-exported attribute. Its discussion +thread spent considerable effort on whether that access should warn, raise, or +do nothing, and on how such enforcement should interact with legitimate +internal access, drawing substantial pushback over adversarial framing, +per-access performance overhead, unreliable warning filters, and breakage of +patterns like pip's: pip has no public API at all, yet still supports tools +such as pip-tools that deliberately import ``pip._internal``. None of that +debate touches the DRY problem this PEP solves. The export bookkeeping problem and the "should Python police access to -internals" problem are separable. This PEP resolves only the former, -leaving module-boundary conventions (a leading underscore on a submodule, -or a private subpackage) to handle the latter. Those conventions already -work, and already ship in every library discussed in the thread. +internals" problem are separable. This PEP resolves only the former, leaving +module-boundary conventions (a leading underscore on a submodule, or a private +subpackage) to handle the latter. Those conventions already work, and already +ship in every library discussed in the thread. -If runtime enforcement is wanted later, a separate proposal can layer it -on top of an accurate, non-duplicated ``__all__``, without entangling it -with the syntax that produces that ``__all__`` in the first place. +If runtime enforcement is wanted later, a separate proposal can layer it on top +of an accurate, non-duplicated ``__all__``, without entangling it with the +syntax that produces that ``__all__`` in the first place. Backwards Compatibility ======================= ``export`` is a soft keyword, following the same approach as ``match``, -``case``, and ``type``. Python treats it specially only in the one -position where ``import`` is otherwise required: immediately after ``from -``. Existing code that uses ``export`` as a variable, function, -parameter, or module name keeps working unchanged, including the unusual -but valid case of a module literally named ``export``. +``case``, and ``type``. Python treats it specially only in the one position +where ``import`` is otherwise required: immediately after ``from ``. +Existing code that uses ``export`` as a variable, function, parameter, or +module name keeps working unchanged, including the unusual but valid case of a +module literally named ``export``. -``from ... export ...`` only affects ``__all__``, which every Python -version already understands. Libraries that support versions before this -feature lands can write both forms, and drop the older one once their -minimum supported version catches up: +``from ... export ...`` only affects ``__all__``, which every Python version +already understands. Libraries that support versions before this feature lands +can write both forms, and drop the older one once their minimum supported +version catches up: .. code-block:: python @@ -563,20 +558,20 @@ This PEP has no known security implications. How to Teach This ================= -Documentation should teach ``from export `` as part of a -named layout, the **hub-and-internals** pattern, not as an isolated -statement. A package following this pattern has two kinds of module: +Documentation should teach ``from export `` as part of a named +layout, the **hub-and-internals** pattern, not as an isolated statement. A +package following this pattern has two kinds of module: -* One **hub** module, typically ``__init__.py`` (a package can have more - than one, such as ``numpy.typing``), whose only job is gathering names - from internal modules and re-exporting them. A hub module contains - ``export`` statements and nothing else that touches ``__all__``; it - never declares ``__all__`` directly, since ``export`` builds it. +* One **hub** module, typically ``__init__.py`` (a package can have more than + one, such as ``numpy.typing``), whose only job is gathering names from + internal modules and re-exporting them. A hub module contains ``export`` + statements and nothing else that touches ``__all__``; it never declares + ``__all__`` directly, since ``export`` builds it. * Any number of **internal** modules, conventionally named with a leading underscore or nested under a leading-underscore subpackage, holding the - actual implementation. Internal modules declare no ``__all__``: they - aren't meant for direct import by users, so there's nothing for - ``__all__`` to curate. + actual implementation. Internal modules declare no ``__all__``: they aren't + meant for direct import by users, so there's nothing for ``__all__`` to + curate. .. code-block:: python @@ -599,19 +594,18 @@ statement. A package following this pattern has two kinds of module: ... This gives one rule to teach: *if a name needs to reach users, write one -``export`` statement for it in the hub; everything else stays unexported -by default.* The whole package declares its public layout in exactly one -place, built from statements that would exist anyway, to make the names -available at all. +``export`` statement for it in the hub; everything else stays unexported by +default.* The whole package declares its public layout in exactly one place, +built from statements that would exist anyway, to make the names available at +all. -Style guides that currently recommend ``from import as -`` for re-exports can point to ``export`` instead. ``export`` covers -conditional re-exports too, such as picking a platform-specific -implementation (see `Specification`_). Direct ``__all__`` manipulation -remains available, and still necessary, outside the hub-and-internals -pattern, for names discovered programmatically at runtime rather than -through a single import statement, such as a loop that registers -plugins. +Style guides that currently recommend ``from import as `` +for re-exports can point to ``export`` instead. ``export`` covers conditional +re-exports too, such as picking a platform-specific implementation (see +`Specification`_). Direct ``__all__`` manipulation remains available, and still +necessary, outside the hub-and-internals pattern, for names discovered +programmatically at runtime rather than through a single import statement, such +as a loop that registers plugins. Reference Implementation @@ -630,55 +624,87 @@ Alternative surface syntax This PEP considered two other spellings for the re-export statement: -* ``export from ``, mirroring ECMAScript's ``export ... - from ...``. Rejected because it puts the name before the module, - reversing the order every Python import statement uses, for no benefit - beyond matching another language's convention. -* ``export from import ``, prefixing an ordinary ``from - ... import ...`` statement with ``export``. An earlier draft of this - proposal used this form. Rejected in favor of ``from export - ``, because a leading ``export`` in front of a complete import - statement reads as two verbs for one action, and because replacing - ``import`` in place keeps the keyword's special-cased position to one - spot in the grammar, rather than requiring the parser to recognize - ``export`` as a prefix before several statement kinds. +* ``export from ``, mirroring ECMAScript's ``export ... from + ...``. Rejected because it puts the name before the module, reversing the + order every Python import statement uses, for no benefit beyond matching + another language's convention. +* ``export from import ``, prefixing an ordinary ``from ... + import ...`` statement with ``export``. An earlier draft of this proposal + used this form. Rejected in favor of ``from export ``, because + a leading ``export`` in front of a complete import statement reads as two + verbs for one action, and because replacing ``import`` in place keeps the + keyword's special-cased position to one spot in the grammar, rather than + requiring the parser to recognize ``export`` as a prefix before several + statement kinds. + + +Open Issues +=========== + +Should ``export`` warn on a non-list ``__all__``? +------------------------------------------------- + +`Specification`_ silently converts a non-list ``__all__`` into a list before +appending to it. Guido van Rossum's parallel proposal for PEP 844's ``@public`` +decorator instead raises a visible ``DeprecationWarning`` when it finds a +non-list ``__all__``, while still supporting the value indefinitely (`comment +`__). + +The open question: should ``export`` do the same and warn when it has to +convert a non-list ``__all__``, or stay silent and leave that to linters, such +as ruff's ``PLE0605``? Acknowledgements ================ -This PEP grew out of discussion on :pep:`842`, particularly -contributions from Peter Bierma, Alex Grönholm, Guido van Rossum, Barry -Warsaw, and Hugo van Kemenade, who supplied the real-world examples of -re-export breakage that ground this proposal in concrete libraries -rather than hypotheticals. +This PEP grew out of discussion on :pep:`842`, particularly contributions from +Peter Bierma, Alex Grönholm, Guido van Rossum, Barry Warsaw, and Hugo van +Kemenade, who supplied the real-world examples of re-export breakage that +ground this proposal in concrete libraries rather than hypotheticals. + + +Footnotes +========= + +.. [#pragprog] Andrew Hunt and David Thomas, *The Pragmatic Programmer* + (Addison-Wesley, 1999). Change History ============== +* 23-Aug-2026 + + - Consolidated the ``__all__``-normalization pseudocode into a single copy in + `Specification`_, instead of repeating it three times. + - Added an Open Issues section asking whether ``export`` should raise a + ``DeprecationWarning`` on a non-list ``__all__``. + - Cited the canonical definition of DRY. + - Restricted ``export`` to module level. + * 22-Aug-2026 - - Resolved the wildcard-form open question in favor of matching - ``import *`` exactly, including its no-``__all__`` fallback; removed - the now-resolved Open Issues section. + - Resolved the wildcard-form open question in favor of matching ``import *`` + exactly, including its no-``__all__`` fallback; removed the now-resolved + Open Issues section. - Made the ``__all__``-creation rules in "Interaction with ``__all__``" - explicit, and added an example showing ``__all__ +=`` after an - ``export`` statement. - - Noted that ``export`` is usable anywhere ``import`` is, with no - restriction of its own, and called out ``if typing.TYPE_CHECKING:`` - re-exports as an intended use case for stub-only packages. + explicit, and added an example showing ``__all__ +=`` after an ``export`` + statement. + - Noted that ``export`` is usable anywhere ``import`` is, with no restriction + of its own, and called out ``if typing.TYPE_CHECKING:`` re-exports as an + intended use case for stub-only packages. * 13-Aug-2026 - - Reworded the public/implementation layout description in the - Abstract, since "flat tree" was self-contradictory. - - Removed an incorrect claim that a missing ``__all__`` costs - ``dir()`` cleanliness; ``dir()`` does not consult ``__all__``. + - Reworded the public/implementation layout description in the Abstract, + since "flat tree" was self-contradictory. + - Removed an incorrect claim that a missing ``__all__`` costs ``dir()`` + cleanliness; ``dir()`` does not consult ``__all__``. Copyright ========= -This document is placed in the public domain or under the -CC0-1.0-Universal license, whichever is more permissive. +This document is placed in the public domain or under the CC0-1.0-Universal +license, whichever is more permissive.