Skip to content

fix(python): resolve annotations through TYPE_CHECKING imports (#1277) - #2700

Merged
DeusData merged 5 commits into
mainfrom
fix/py-type-checking-imports-v2
Oct 19, 2026
Merged

DeusData merged 5 commits into
mainfrom
fix/py-type-checking-imports-v2

Conversation

@DeusData

@DeusData DeusData commented Oct 18, 2026 •

Copy link
Copy Markdown
Owner

What

The import that an annotation's type comes from usually sits under a module-level if TYPE_CHECKING:, to
avoid an import cycle:

from typing import TYPE_CHECKING
if TYPE_CHECKING:
    from homeassistant.core import HomeAssistant

async def async_setup(hass: HomeAssistant) -> bool:
    hass.async_create_task(...)        # needs HomeAssistant to resolve

parse_python_imports walked only the module's direct statements, so that import was never recorded. Every
call through such an annotation fell to a name guess or got no edge at all.

  • Extraction records the imports of a module-level if TYPE_CHECKING: / if <module>.TYPE_CHECKING:
    consequence block, flagged CBMImport.type_checking_only.
  • The elif/else branches are runtime alternatives and stay unrecorded, as every other compound statement's
    imports do. A name such a branch binds again is bound at runtime, so its TYPE_CHECKING import is not an
    annotation-only import, and it stays unrecorded too. This is the compat shape, e.g.
    else: try: from airflow.sdk.log import mask_secret. A star import there covers every name of the
    block. Those names keep exactly the behaviour they have today.
  • One rule for every CALLS import-map builder: cbm_import_binds_runtime_name(), which is false for a
    flagged import. So no CALLS import map binds an annotation-only name, and a runtime call through one gets
    no edge.
  • The annotation consumers use the flagged imports: the cross-file base / field-type resolution and the
    Python LSP. The LSP binds them in an annotation-only frame, consulted by annotation resolution after the
    ordinary scope chain.

No index-format change.

Evidence

Base main 1346ac4 vs this change, 26 Python repos (manifest v1), production builds, macOS arm64,
--repeat 2, every repeat byte-identical.

type base this PR +added -removed
CALLS 1,355,020 1,371,605 16,611 26
TESTS 511,519 521,807 10,289 1
IMPORTS 477,476 486,103 8,627 0
CALL_REFERENCE 26,918 26,941 23 0
USAGE 1,672,278 1,672,256 0 22
INHERITS 78,598 78,597 0 1
  • New CALLS (16,611): almost all LSP-resolved. The strategies are lsp_method 16,301,
    lsp_method_union 118, lsp_operator_dunder 111 and lsp_callable_alias 12, plus a few name
    strategies (86). The biggest share is home-assistant (14,430), whose integrations import HomeAssistant,
    ConfigEntry and friends under TYPE_CHECKING. Then scrapy 634, airflow 379, sqlalchemy 219, poetry 181,
    sentry 154 and mypy 136. TESTS follow from the new CALLS.
  • New IMPORTS (8,627): the TYPE_CHECKING imports themselves, from the file to the imported module or
    symbol, as for runtime imports.
  • Removed CALLS (26): 13 are retargeted on the same line to an LSP-typed target. The other 13 were wrong
    name guesses that are now gone: event.get_event_type() and dag_maker.create_dagrun() had been bound
    to same-named module functions, and an openlineage Dataset(...) to an unrelated task-sdk Dataset.
  • USAGE -22: 16 of them come back as CALL_REFERENCE with the same source and target. The other 6 were
    cross-component name guesses.
  • INHERITS -1: sentry's SentryPrinter(honcho.printer.Printer) has an external base, and the in-repo
    Printer it was bound to was wrong.

Judged precision of the additions

Seeded samples (seed 20261017), each judged against its call line and target definition: 50/50 correct.

  • home-assistant CALLS: 20/20. 96% of its 14,430 new edges are hass.config_entries.<method>(...) reaching
    ConfigEntries.<method>, because HomeAssistant.config_entries is annotated with a type that core.py
    imports under TYPE_CHECKING. The rest go through hass.auth (AuthManager), hass.http
    (HomeAssistantHTTP), the recorder (Recorder) and annotated entity parameters. A spot check of that
    remainder (12 edges) is also correct.
  • CALLS in the other 25 repos: 20/20. Annotated receivers reach their class's own or inherited method:
    ansible self._loader reaches DataLoader, scrapy self.stats reaches StatsCollector, mypy builder
    reaches IRBuilder / LowLevelIRBuilder, airflow dag_maker reaches DagMaker, sympy Expr reaches
    Basic methods, and so on.
  • TESTS: 10/10. Each links a test function to the ConfigEntries method it calls.

Tests

  • python_type_checking_imports_flagged_issue1277 (extraction): TYPE_CHECKING and typing.TYPE_CHECKING
    imports are recorded and flagged. A runtime import is not flagged, and an else branch and an ordinary
    if stay unrecorded.
  • python_type_checking_rebound_in_else_unrecorded_issue1277 (extraction): a name the else branch binds
    again under try/except is not recorded. A star import in an elif covers every name, and a pure
    annotation import stays flagged.
  • python_type_checking_typed_field_issue1277{,_parallel} (pipeline): the reporter's typed-field chain
    with both imports under TYPE_CHECKING resolves through the LSP. Controls: there is no import_map and no
    LSP CALLS edge through the annotation-only name.
  • All RED without the fix.
  • Affected suites (complexity, extraction, extraction_imports, grammar_imports, registry, pipeline,
    cross_repo, py_lsp, py_lsp_stress, py_lsp_scale, lsp_resolution_probe, edge_imports, edge_types_probe,
    convergence_probe, matrix_known_classes, parallel, incremental, integration) pass.

Part of #1277 (the TYPE_CHECKING half).

The second half of #1277: the reporter's typed-field chain with both
annotation imports under `if TYPE_CHECKING:` (the usual shape when the
import would cycle), through `typing.TYPE_CHECKING` too. The call resolves
by the LSP as with runtime imports, and a runtime call through a
TYPE_CHECKING-only name gets no CALLS edge at all. The extraction test reads
the new CBMImport.type_checking_only flag, declared here and not yet set.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The import an annotated field's type comes from usually sits under a
module-level `if TYPE_CHECKING:` (to avoid an import cycle).
parse_python_imports walked only the module's direct statements, so that
import was never recorded and the reporter's chain
trainer.strategies.process_batch() lost its type exactly where real code
writes it.

- Extraction records the imports of a module-level `if TYPE_CHECKING:` /
  `if <module>.TYPE_CHECKING:` consequence block, flagged
  CBMImport.type_checking_only. elif/else branches stay unrecorded, as
  every other compound statement's imports do.
- One rule for every CALLS import-map builder:
  cbm_import_binds_runtime_name() (false for a flagged import). The
  sequential build_import_map applies it in its fast and slow paths; the
  shared cbm_pxc_build_import_map (= _ex(..., false)) applies it in both
  loops, so no CALLS import map ever binds an annotation-only name.
- The annotation consumers take cbm_pxc_build_import_map_ex(..., true):
  the cross-file base/field-type resolution and the Python LSP dispatch.
  The sequential dispatch always takes it; the parallel one builds that
  map only for a file that records a flagged import and otherwise passes
  its CALLS map, which is identical there by construction.
- The Python LSP binds those names into an annotation-only frame consulted
  by annotation resolution after the ordinary scope chain: a runtime call
  through such a name never resolves.

Tests: python_type_checking_imports_flagged_issue1277 (extraction),
python_type_checking_typed_field_issue1277(_parallel) (pipeline, both
venues; controls: no import_map and no LSP CALLS edge through the name).

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
… import (#1277)

The airflow compat shape imports a name under `if TYPE_CHECKING:` and again
in the else branch, usually inside try/except ImportError: the name is bound
at runtime, so its TYPE_CHECKING import is not annotation-only and must stay
unrecorded, as the runtime import itself does. A star import in an elif or
else branch may bind any name. A pure annotation import stays flagged.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
…ecorded (#1277)

The compat shape imports a name under `if TYPE_CHECKING:` and again in the
else branch, usually inside try/except ImportError. The name is bound at
runtime there, so its TYPE_CHECKING import is no annotation-only import:
recording it flagged made every CALLS import map skip the name and lost the
calls and bases that bind through the runtime import (airflow mask_secret,
OpenLineageRedactor(SecretsMasker)). The imports of the elif/else branches
are collected (any depth, iteratively), every TYPE_CHECKING import whose
local name they bind is dropped with them, and a star import there drops the
whole block: those names keep the behaviour they had before #1277.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The memory-core linter counts raw allocator calls per file and fails when a
file grows. The TYPE_CHECKING change freed the copied local name of an
annotation-only import with a new raw free in two import-map builders.
build_import_map now asks the predicate on the name's span in the edge
properties before copying it (cbm_imports_local_span_binds_runtime_name,
which cbm_imports_local_binds_runtime_name now wraps), and the cross-LSP
builder routes an annotation-only name through its existing no-value path.
Same behaviour, no new raw allocator site.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
@DeusData

Copy link
Copy Markdown
Owner Author

Lint fix (3ead775): the memory-core linter failed because this change added a raw free in two
import-map builders (pass_calls.c, pass_lsp_cross.c). build_import_map now checks the predicate on
the name's span in the edge properties before copying it (cbm_imports_local_span_binds_runtime_name,
which cbm_imports_local_binds_runtime_name now wraps). The cross-LSP builder sends an annotation-only
name down its existing no-value path. Same behaviour, no new raw allocator site.
make -f Makefile.cbm lint-memory-core passes on the branch merged with main.

The graph cannot change from this commit, so the corpus measurement in the description still holds.

@DeusData
DeusData merged commit e6b47a3 into main Oct 19, 2026
28 checks passed
DeusData added a commit that referenced this pull request Oct 19, 2026
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant