Repository navigation
Build whole models from mathspec (whole-model-only) - #958
Merged
Merged
Conversation
Merging this PR will not alter performance
Comparing Footnotes
|
Build cost — v1 vs legacyv1 build peak & time relative to legacy, on this commit — not a comparison against master (that is CodSpeed).
Full table (time + peak, mean)📊 Interactive plots + CSV: download the semantics-report-v1-vs-legacy artifact from this run. Report-only · not a gate · refreshed on every push · obsolete once legacy is dropped. |
FabianHofmann
marked this pull request as ready for review
September 18, 2026 07:45
This was referenced Sep 19, 2026
…pec 0.4 Core hooks (spec stamps, add_spec/from_spec, netcdf spec block, expression stamps) in master's shape; linopy.spec rewritten for mathspec 0.4 nodes, joins, missing:, ordered:, conditions:. Phase 4 fixes 28 spec tests.
…trict; fold assumptions and groups)
FabianHofmann
force-pushed
the
spec-whole-model
branch
from
October 26, 2026 15:38
fd3c8d5 to
f168f8f
Compare
…on replaced objective or undeclared SOS
Fold duplicated helpers (prefixing, netcdf filters, lookups, zeros, gathers, as_expression), drop dead checks and single-use wrappers; align only dense input. Test string parameters given as rows.
…y handles, KeyError for unknown names Fold the binary-operator branches of evaluate into one table, drop the redundant source-key cache and guards in attach, and build the row presence mask with Index.isin.
Keep every parameter unfilled in ModelSpec.parameters and fill on read; drop the .spec stamp, presence dataset, add_spec, lower_text and the file-name sniffing of str specs. Public surface is ModelSpec, NamedExpression, SpecDataError, Declaration and attach. Notebook updated.
5 tasks done
from_spec writes raw parameters and relations into model.parameters; linopy.spec.evaluate and linopy.spec.parameter replace ModelSpec. Object parameters round-trip netcdf with NaN holes; grid sources accept NaN as a missing row; sets, extension arrays and named value columns are accepted; str file names are refused with a Path hint. Docs: spec-data page.
A hole in a refused parameter is excused where where: masks, a masked variable, an absent row, an unfilled shift edge or a failed lookup make the term absent anyway. build and evaluate share one check that names the parameter, the declaration and the missing labels.
This was referenced Oct 27, 2026
…dth, unshadow SHIFTED
…True Model.from_spec gains store=False; evaluate takes sources= and otherwise reads model.parameters through the same attach path. Drop linopy.spec.parameter.
…th mathspec join semantics
4 of 5 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Note
The following content was generated by AI.
Changes proposed in this Pull Request
A mathspec spec (a YAML declaration of a linear model) builds a whole linopy model from
linopy/spec/. mathspec (>=0.4,<0.5, from PyPI, Python >= 3.12) is the optional extralinopy[spec]and imported lazily, soimport linopynever pulls it in.This branch was re-ported onto master (after #1041, #1042, #1051–#1059), migrated to mathspec 0.4 and leaned down. The older 67-commit history is kept under the tag
spec-whole-model-v1. The simplification from #1068 is merged into this branch.API
Model.from_spec(spec, sources, **model_kwargs)builds variables, SOS, constraints, the objective and named expressions with a variable term.specis aPath, YAML text, dict ormathspec.Spec. A one-linestrending in.yaml/.ymlraises aValueErrorthat asks for aPath.model.parameters. Every parameter (unfilled, NaN at holes) and every relation is stored under its declared name.assign_coords,copyand the netcdf round trip therefore carry the spec data. There is nomodel.specaccessor and no stamp on variables or constraints.(spec, model).linopy.spec.evaluate(spec, model, *names)folds named expressions over the solution and the parameters.linopy.spec.parameter(spec, model, name)returns filled values with the declared dtype.linopy.spec.__all__isSpecDataError,evaluate,parameter.mathspec.typeset(spec, fmt)andtypeset_line.Model.from_spec(path, read_netcdf(f).parameters)rebuilds without the original sources.linopy.testing.synthetic_sourcestakes every spec formfrom_spectakes.Sources
missing:policy. Long tables still refuse NaN; a long table may carry its values in a column named after the parameter.Mappingruns one query per parameter.mathspec 0.4 semantics
sum(x, over=d, by=rel[c]),at(x, by=rel[c]), sums over several axes. Grouped sums on terms use master'sRelationgroup-by.missing:per parameter and relation (refuseddefault,neutral, a literal,absent). Absence travels as a mask, never as NaN.missing: refusedonly raises for holes a live row demands. A hole is excused under awhere:mask, a masked variable, anabsentrow of another parameter, an unfilled shift edge or a failed lookup. Build andevaluateshare one check that names the parameter, the declaration and the missing labels.ordered:,conditions:and thedual(c)sign convention are supported.Changes for all models
''.['a', 1]) raise on write instead of being written as strings.read_netcdfkeeps parameter coordinates that no data variable uses.Limits
SpecDataError. Follow-up: spec: full mathspec relation joins and quadratic constraints (follow-up to #958) #1063 (needed by mathspec'spypsa.yaml).Docs, CI, benchmarks
doc/spec-data.rst(source contract, SQLite example, holes, rebuild, variants via mathspec). Notebookexamples/building-models-from-specs.ipynb,doc/api.rst, user guide and release notes.operators/,ports/and the top level. On Python 3.11, where mathspec is not installable, the spec tests skip.benchmarks/models/spec_commitment.py(commitment.yamlplus asum_by-shaped relation case). codspeed and benchmark-smoke install nospecextra, so these benchmarks skip in CI.API review findings deferred to issues
Updating data and re-solving (editing
model.parameterschangesevaluatebut not the model); applying a spec to an existing model and composing programs; bare relations and sets derived from data; sparse data densified; data-only named expressions not inmodel.expressions; lazy per-parameter reads.Local verification
Checklist
AGENTS.md).doc.doc/release_notes.rstof the upcoming release is included.