B023: don't flag a function that is only called inside the loop - #567
Conversation
There was a problem hiding this comment.
Pull request overview
Updates B023 to exempt loop-defined functions referenced only through direct calls within the loop.
Changes:
- Adds direct-call reference analysis for loop-defined functions.
- Adds regression and escape-case coverage.
- Documents the behavior change.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
bugbear.py |
Implements immediate-call detection. |
tests/eval_files/b023.py |
Adds B023 evaluation cases. |
README.rst |
Updates the changelog. |
Suppressed comments (2)
bugbear.py:1160
- These checks still accept direct calls from nested functions. If
f()is called once at loop level and also from a storedwrapper, bothNamenodes are call targets insideast.walk(loop_node), sofis exempted even thoughwrapper()can invoke it after the loop. Track each reference's lexical owner and reject calls reached through another nested function (while handling self-recursion separately if desired).
if (
not isinstance(node.ctx, ast.Load)
or id(node) not in call_targets
or id(node) not in in_loop
):
bugbear.py:1150
ast.walk(loop_node)is not sufficient proof that a call occurs in the defining iteration: it includes the loop'selsesuite, and it also accepts a call textually before thedef(which invokes the previous iteration's binding). Both cases let the function outlive its defining iteration but are marked safe. Excludeorelseand require each accepted call to be reached after the matching definition on every relevant control-flow path.
root = self.node_stack[0] if self.node_stack else loop_node
in_loop = {id(node) for node in ast.walk(loop_node)}
call_targets = {
id(node.func)
for node in ast.walk(root)
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| if ( | ||
| isinstance(node, (ast.AsyncFunctionDef, ast.FunctionDef)) | ||
| and not node.decorator_list | ||
| and node.name in immediately_called | ||
| ): |
| candidates = { | ||
| node.name | ||
| for node in ast.walk(loop_node) | ||
| if isinstance(node, (ast.AsyncFunctionDef, ast.FunctionDef)) | ||
| and not node.decorator_list # a decorator may stash the original |
| call_targets = { | ||
| id(node.func) | ||
| for node in ast.walk(root) | ||
| if isinstance(node, ast.Call) and isinstance(node.func, ast.Name) | ||
| } |
cooperlees
left a comment
There was a problem hiding this comment.
LGTM - Thanks for this.
I think copilot has found some nice things to polish up - Feel free to state why it's wrong tho if it is.
|
Thanks — the Copilot points hold up, and four of them are the same root cause: I proved "called in this iteration" with Concretely:
I will push a revision with regression cases for each of the five and report back. If you would rather see this as a narrower rule — for example exempting only a direct call in the loop body at statement level — say so and I will cut it to that instead. |
|
I'm happy to start simpler, but if you're happy to do all five go for it. Both ways are an imrpovement so will take what ever you have time for :) ... thanks! |
check_for_b023 warns whenever a function defined in a loop closes over the loop variable, but a function whose every reference is a direct call in the loop body cannot outlive the iteration it was defined in, so the value it closes over is the one the author meant. The existing safe_functions notion covered only a fixed set of shapes - filter/map/reduce, a key= argument, and 'return lambda: x'. It is replaced by the rule the maintainers described on the two issues: warn when the name escapes the loop or is referenced as anything other than a direct call, and stay silent otherwise. Decorated definitions keep warning, since a decorator can store the function. Fixes PyCQA#468 Fixes PyCQA#380
Addresses the five review points. The exemption is now granted only when every reference to the name, in the scope that holds the loop, is a call that runs in the same iteration as the definition: * only a plain `def` qualifies. Calling an `async def` builds a coroutine and calling a generator function builds a generator, so the body -- and the read of the loop variable -- is deferred. * a name that is bound anywhere else in the scope is skipped, so a reference to an unrelated binding of the same identifier can no longer decide the outcome. * the search covers the loop body only. The `else` suite runs after the loop, and a call placed above the `def` invokes the binding the previous iteration left behind. * a reference reached through a nested function, lambda or generator expression disqualifies: that body decides when the call happens. * a definition nothing refers to is reported, because its name is still bound after the loop. * the parent links, name uses and call targets of a scope are built once and cached, instead of two walks of the scope per loop.
86c99c5 to
b3cd69b
Compare
|
Went with all five. Also rebased onto main, so the conflict is gone. The rewrite ties the exemption to the definition rather than to the identifier. A name is exempted only when it has at least one reference and every reference in the scope holding the loop is a call that runs in the same iteration as the
One case the rewrite surfaced that none of us had listed: a definition nothing refers to was being exempted, because "every reference is a call" is vacuously true on an empty set. Measured
The nested-function case (
If any of the five reads as over-strict to you, say which and I will loosen that one rather than the set. |
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.
Suppressed comments (2)
bugbear.py:1164
- This only considers
defstatements that are direct children of the loop. A function defined and called within anif,try, orwithin the loop still cannot outlive that iteration, but it remains flagged—for example,for i in xs: if cond: def f(): return i; f(). Please collect candidates through immediate control-flow suites while still stopping at nested lexical/deferred scopes.
for statement in body:
if (
isinstance(statement, ast.FunctionDef)
and not statement.decorator_list
and not _defines_a_generator(statement)
):
bugbear.py:1183
globalandnonlocaldeclarations are not represented asast.Namestores, so_is_reboundmisses them. Inside a function,global f; for i in xs: def f(): return i; f()is therefore exempted even though each definition is stored globally and can be called after the loop with the finali. Please exclude candidates whose binding is declaredglobalornonlocalin the enclosing scope.
# matching a reference by its identifier alone is only sound while
# the name has exactly one binding in this scope
if len(definitions) != 1 or _is_rebound(name, names_used, parents):
continue
Fixes #468. Fixes #380.
check_for_b023warns whenever a function defined in a loop closes over the loop variable. But a function whose every reference is a direct call in the loop body cannot outlive the iteration it was defined in, so the value it closes over is the one the author meant:The existing
safe_functionsnotion covered only a fixed set of shapes —filter/map/reduce, akey=argument, andreturn lambda: x. This replaces it with the rule you both described on the issues: warn when the name escapes the loop or is referenced as anything other than a direct call, and stay silent otherwise. @jakkdl spelled it out on #380 and @cooperlees agreed with it on #468; I've implemented both halves plus one guard of my own — a decorated definition keeps warning, because a decorator can store the function.Measured
python -m pytest: 79 passed, 2 skipped — before and after, on2155484.Still flagged (the name escapes):
[lambda: i for i in range(3)]out.append(lambda: i)return gfrom inside the loopd[i] = g@staticmethodabove the definitionNo longer flagged (every reference is a direct call):
results.append(g())print(g())tests/eval_files/b023.pygains the new cases in the project's declarative format; the existing expectations are unchanged — the one line that moves in the diff is the same line at a new offset, not a changed expectation.blackandisortare clean.rstcheckreports two unreferenced hyperlink targets inREADME.rstat lines 456 and 463 — that is identical on a clean checkout and unrelated to the change here.AI-assisted (LLM used for drafting); the runs above are mine.