Skip to content

model.base: Protect NamespaceSet attributes from being replaced - #619

Open
zrgt wants to merge 3 commits into
eclipse-basyx:developfrom
rwth-iat:fix/65-namespaceset-attribute-assignment
Open

zrgt wants to merge 3 commits into
eclipse-basyx:developfrom
rwth-iat:fix/65-namespaceset-attribute-assignment

Conversation

@zrgt

@zrgt zrgt commented Aug 28, 2026 •

Copy link
Copy Markdown
Contributor

Attributes like Submodel.submodel_element, Qualifiable.qualifier or
HasExtension.extension hold a NamespaceSet, while the corresponding
__init__ parameters accept any Iterable. Assigning an iterable to
such an attribute after initialization silently replaced the
NamespaceSet with that plain object. The old NamespaceSet remained
registered in namespace_element_sets, so its objects stayed reachable
via get_referable() and friends while being invisible in the
attribute, uniqueness constraints were no longer checked and the
parent of the new objects was never set. The resulting errors, such as
AttributeError: 'list' object has no attribute 'update_nss_from' when
iterating over an ObjectStore, gave no hint about the actual cause.

Assignments to such an attribute are now redirected to the content of
the existing NamespaceSet, which is the behavior a user would expect
from the type of the __init__ parameters:

submodel.submodel_element = [my_property]  # replaces the content of the set
aas.extension = []                         # clears the set

This is done by the reusable NamespaceSetAttribute descriptor, which is
declared once per attribute that holds a NamespaceSet. Hooking only
those attributes, instead of overriding Namespace.__setattr__, leaves
every other attribute write at its original speed. extension and
qualifier are declared on HasExtension and Qualifiable, so they
cover every current and future subclass, including classes defined
outside of this repository. The existing SubmodelElementList.value
setter keeps working unchanged.

The replacement is atomic: if one of the new objects violates a
constraint, the previous content of the set is restored, so a failed
assignment leaves the object untouched.

Fixes #65

Attributes like `Submodel.submodel_element`, `Qualifiable.qualifier` or
`HasExtension.extension` hold a `NamespaceSet`, while the corresponding
`__init__` parameters accept any `Iterable`. Assigning an iterable to
such an attribute after initialization silently replaced the
`NamespaceSet` with that plain object. The old `NamespaceSet` remained
registered in `namespace_element_sets`, so its objects stayed reachable
via `get_referable()` and friends while being invisible in the
attribute, uniqueness constraints were no longer checked and the
`parent` of the new objects was never set. The resulting errors, such as
`AttributeError: 'list' object has no attribute 'update_nss_from'` when
iterating over an `ObjectStore`, gave no hint about the actual cause.

`Namespace.__setattr__` now redirects such assignments to the content of
the existing `NamespaceSet`, which is the behavior a user would expect
from the type of the `__init__` parameters. This is done centrally in
`Namespace` instead of via a property per attribute, so that it also
applies to the `NamespaceSet` attributes of classes outside of this
repository. The replacement is atomic: if one of the new objects
violates a constraint, the previous content of the set is restored.

Fixes #65
@zrgt
zrgt requested a review from s-heppner August 28, 2026 16:33
@s-heppner

Copy link
Copy Markdown
Member

Depends on #661, since otherwise our Registry service would be broken.

@s-heppner s-heppner left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This __setattr__ override runs on every attribute write of every Namespace subclass, and that has a measurable cost. Object construction is about 3× slower than on the merge base:

Benchmark Merge base This PR
Property("P", Int, 1) ~6 µs ~18 µs
Submodel with 20 Propertys ~135 µs ~390 µs
import timeit
from basyx.aas import model

f = lambda: model.Submodel(
    "urn:sm",
    submodel_element=[model.Property(f"P{i}", model.datatypes.Int, i) for i in range(20)],
)
print(min(timeit.repeat(f, number=500, repeat=5)) / 500 * 1e6, "µs")

This affects every deserialization and every backend that builds objects, so large models will feel it. Most of these writes never touch a NamespaceSet attribute.

Could we hook only the attributes that actually hold a NamespaceSet? One option is a property (or a small reusable descriptor) per attribute, like the existing SubmodelElementList.value setter and like we agreed in #65. Then normal attribute writes keep their current speed.

:raises ValueError: If one of the given items already belongs to another namespace
"""
previous: List[_NSO] = list(self)
self.clear()

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

assign() clears the set before it reads items. If items is a lazy iterable over the same set, iterating it fails:

sm = model.Submodel("urn:sm", submodel_element=[
    model.Property("P1", model.datatypes.Int),
    model.Property("P2", model.datatypes.Int),
])
sm.submodel_element = filter(lambda x: x.id_short == "P1", sm.submodel_element)
# RuntimeError: dictionary changed size during iteration

The self.clear() leaves the set unchanged, which is good, but we'd lose functionality that currently works on develop.
One could argue that the actual set should not be accessed, but filtering a set into itself (like above) is a pretty common thing to write imo. Reading items into a list first fixes it:

Suggested change
self.clear()
new_items: List[_NSO] = list(items)
previous: List[_NSO] = list(self)
self.clear()
try:
for item in new_items:
self.add(item)

A test with a generator or filter() over the set itself would cover this case.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in a2a6045: assign() now does list(items) before self.clear().

Added test_assignment_of_lazy_iterable_over_the_same_set, which covers both your filter() example and a generator expression over the set.


Generated by Claude Code

Protecting the NamespaceSet attributes via `Namespace.__setattr__` ran on
every attribute write of every `Namespace` subclass. Nearly none of these
writes target a NamespaceSet attribute, yet each one paid for a
Python-level `__setattr__` call: constructing a `Property` took 8.2 µs
instead of 2.6 µs, a `Submodel` with 20 `Property`s 187 µs instead of
66 µs (CPython 3.12). Every deserialization and every backend that builds
objects carries that cost.

`NamespaceSetAttribute` hooks only the attributes that actually hold a
NamespaceSet, so every other write keeps its original speed. What remains
is one descriptor call per NamespaceSet attribute in `__init__`, where a
NamespaceSet was constructed anyway. `OrderedNamespaceSetAttribute` is the
same descriptor for attributes holding an `OrderedNamespaceSet`; it only
narrows the type of the attribute.

`NamespaceSet.assign()` cleared the set before it read the given iterable.
A lazy iterable over that same set, as produced by
`filter(..., submodel.submodel_element)`, therefore failed while being
iterated. It is now consumed before the set is cleared.
@zrgt

zrgt commented Oct 22, 2026

Copy link
Copy Markdown
Contributor Author

You were right, and the slowdown is bigger than the __setattr__ body suggests: Property("P", Int, 1) performs 30 attribute writes, none of which touches a NamespaceSet.

a2a6045 drops the __setattr__ override for a reusable descriptor, NamespaceSetAttribute, declared once per attribute that holds a NamespaceSet — on HasExtension.extension and Qualifiable.qualifier, so all their subclasses are covered, and on the seven per-class attributes in submodel.py. OrderedNamespaceSetAttribute is the same descriptor with a narrowed __get__ type.

Your benchmark, CPython 3.12, best of 7:

Benchmark Merge base __setattr__ Descriptor
Property("P", Int, 1) 2.57 µs 8.17 µs 2.73 µs
Submodel with 20 Propertys 65.5 µs 187 µs 68.2 µs

What is left is one descriptor call per NamespaceSet attribute in __init__, where a NamespaceSet was constructed anyway.

Two side effects worth noting:

Checks on sdk: 278 tests pass (8 skipped — JSON/XSD schemas and CouchDB not available locally), mypy and ruff clean, Sphinx builds with -n -W, coverage 89 %. compliance_tool and server mypy/ruff clean, their tests unchanged (the two server test errors reproduce on the merge base).


Generated by Claude Code

@zrgt

zrgt commented Oct 22, 2026

Copy link
Copy Markdown
Contributor Author

CI on a2a6045 is green except server-test (3.12) (and server-test (3.11), cancelled by fail-fast). That failure is not this PR's:

  • All 124 errors are TypeError: check() got an unexpected keyword argument 'attribute', raised from basyx/aas/model/_string_constraints.py in site-packages — the job tests the server against the PyPI SDK release, not the local one. Nothing in this PR touches _string_constraints.
  • develop itself fails identically: run 39418262596 on 2773ac0, same 124 errors, same message.
  • The server-test job does not exist in this branch's .github/workflows/pr.yml; it reaches the PR through the merge with develop.

This is #662: the job checks out shallow, so setuptools_scm versions the local SDK below the server's basyx-python-sdk>=1.0.0 constraint and pip install .[dev] replaces it with the release. The documented fix is fetch-depth: 0 on that job's checkout, as the other jobs already do. No fix PR exists yet, and porting a workflow change for a job this branch does not even define would widen this PR into unrelated CI work, so I left it alone.

I also did not spend a re-run on it: the cause is deterministic, and develop being red the same way is stronger evidence than a re-run would be.


Generated by Claude Code

@zrgt
zrgt requested a review from s-heppner October 22, 2026 00:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add setter-functions for object attributes that consist of NamespaceSets

2 participants