Repository navigation
Conversation
Attributes like `Submodel.submodel_element`, `Qualifiable.qualifier` or `HasExtension.extension` hold a `NamespaceSet`, while the corresponding `__init__` parameters accept any `Iterable`. Assigning an iterable to such an attribute after initialization silently replaced the `NamespaceSet` with that plain object. The old `NamespaceSet` remained registered in `namespace_element_sets`, so its objects stayed reachable via `get_referable()` and friends while being invisible in the attribute, uniqueness constraints were no longer checked and the `parent` of the new objects was never set. The resulting errors, such as `AttributeError: 'list' object has no attribute 'update_nss_from'` when iterating over an `ObjectStore`, gave no hint about the actual cause. `Namespace.__setattr__` now redirects such assignments to the content of the existing `NamespaceSet`, which is the behavior a user would expect from the type of the `__init__` parameters. This is done centrally in `Namespace` instead of via a property per attribute, so that it also applies to the `NamespaceSet` attributes of classes outside of this repository. The replacement is atomic: if one of the new objects violates a constraint, the previous content of the set is restored. Fixes #65
|
Depends on #661, since otherwise our Registry service would be broken. |
s-heppner
left a comment
There was a problem hiding this comment.
This __setattr__ override runs on every attribute write of every Namespace subclass, and that has a measurable cost. Object construction is about 3× slower than on the merge base:
| Benchmark | Merge base | This PR |
|---|---|---|
Property("P", Int, 1) |
~6 µs | ~18 µs |
Submodel with 20 Propertys |
~135 µs | ~390 µs |
import timeit
from basyx.aas import model
f = lambda: model.Submodel(
"urn:sm",
submodel_element=[model.Property(f"P{i}", model.datatypes.Int, i) for i in range(20)],
)
print(min(timeit.repeat(f, number=500, repeat=5)) / 500 * 1e6, "µs")This affects every deserialization and every backend that builds objects, so large models will feel it. Most of these writes never touch a NamespaceSet attribute.
Could we hook only the attributes that actually hold a NamespaceSet? One option is a property (or a small reusable descriptor) per attribute, like the existing SubmodelElementList.value setter and like we agreed in #65. Then normal attribute writes keep their current speed.
| :raises ValueError: If one of the given items already belongs to another namespace | ||
| """ | ||
| previous: List[_NSO] = list(self) | ||
| self.clear() |
There was a problem hiding this comment.
assign() clears the set before it reads items. If items is a lazy iterable over the same set, iterating it fails:
sm = model.Submodel("urn:sm", submodel_element=[
model.Property("P1", model.datatypes.Int),
model.Property("P2", model.datatypes.Int),
])
sm.submodel_element = filter(lambda x: x.id_short == "P1", sm.submodel_element)
# RuntimeError: dictionary changed size during iterationThe self.clear() leaves the set unchanged, which is good, but we'd lose functionality that currently works on develop.
One could argue that the actual set should not be accessed, but filtering a set into itself (like above) is a pretty common thing to write imo. Reading items into a list first fixes it:
| self.clear() | |
| new_items: List[_NSO] = list(items) | |
| previous: List[_NSO] = list(self) | |
| self.clear() | |
| try: | |
| for item in new_items: | |
| self.add(item) |
A test with a generator or filter() over the set itself would cover this case.
There was a problem hiding this comment.
Fixed in a2a6045: assign() now does list(items) before self.clear().
Added test_assignment_of_lazy_iterable_over_the_same_set, which covers both your filter() example and a generator expression over the set.
Generated by Claude Code
Protecting the NamespaceSet attributes via `Namespace.__setattr__` ran on every attribute write of every `Namespace` subclass. Nearly none of these writes target a NamespaceSet attribute, yet each one paid for a Python-level `__setattr__` call: constructing a `Property` took 8.2 µs instead of 2.6 µs, a `Submodel` with 20 `Property`s 187 µs instead of 66 µs (CPython 3.12). Every deserialization and every backend that builds objects carries that cost. `NamespaceSetAttribute` hooks only the attributes that actually hold a NamespaceSet, so every other write keeps its original speed. What remains is one descriptor call per NamespaceSet attribute in `__init__`, where a NamespaceSet was constructed anyway. `OrderedNamespaceSetAttribute` is the same descriptor for attributes holding an `OrderedNamespaceSet`; it only narrows the type of the attribute. `NamespaceSet.assign()` cleared the set before it read the given iterable. A lazy iterable over that same set, as produced by `filter(..., submodel.submodel_element)`, therefore failed while being iterated. It is now consumed before the set is cleared.
|
You were right, and the slowdown is bigger than the a2a6045 drops the Your benchmark, CPython 3.12, best of 7:
What is left is one descriptor call per Two side effects worth noting:
Checks on Generated by Claude Code |
|
CI on a2a6045 is green except
This is #662: the job checks out shallow, so I also did not spend a re-run on it: the cause is deterministic, and Generated by Claude Code |
Attributes like
Submodel.submodel_element,Qualifiable.qualifierorHasExtension.extensionhold aNamespaceSet, while the corresponding__init__parameters accept anyIterable. Assigning an iterable tosuch an attribute after initialization silently replaced the
NamespaceSetwith that plain object. The oldNamespaceSetremainedregistered in
namespace_element_sets, so its objects stayed reachablevia
get_referable()and friends while being invisible in theattribute, uniqueness constraints were no longer checked and the
parentof the new objects was never set. The resulting errors, such asAttributeError: 'list' object has no attribute 'update_nss_from'wheniterating over an
ObjectStore, gave no hint about the actual cause.Assignments to such an attribute are now redirected to the content of
the existing
NamespaceSet, which is the behavior a user would expectfrom the type of the
__init__parameters:This is done by the reusable
NamespaceSetAttributedescriptor, which isdeclared once per attribute that holds a
NamespaceSet. Hooking onlythose attributes, instead of overriding
Namespace.__setattr__, leavesevery other attribute write at its original speed.
extensionandqualifierare declared onHasExtensionandQualifiable, so theycover every current and future subclass, including classes defined
outside of this repository. The existing
SubmodelElementList.valuesetter keeps working unchanged.
The replacement is atomic: if one of the new objects violates a
constraint, the previous content of the set is restored, so a failed
assignment leaves the object untouched.
Fixes #65