Skip to content

feat(restrict): window a model's flattened matrices - #1036

Open
olivier-lacroix wants to merge 7 commits into
PyPSA:masterfrom
olivier-lacroix:feat/restrict-matrix-windows
Open

olivier-lacroix wants to merge 7 commits into
PyPSA:masterfrom
olivier-lacroix:feat/restrict-matrix-windows

Conversation

@olivier-lacroix

@olivier-lacroix olivier-lacroix commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

Closes #768. Interested in having rolling horizon patterns in linopy.

  • I used the original proposition in terms of method naming. This can be changed to whatever people prefer.
  • My first attempt at this sliced each container individually. This allowed to work on the windowed model as on a normal model, but required a lot of code, and was kind of slowish.

Note

The following content was generated by AI.

What

Model.restrict returns a WindowModel backed by a slice of the parent's
flattened A, b, c, lb and ub. A window is a row slice plus a column
range; terms whose variables fall outside the window are folded into the RHS at
their actual matrix coefficient. No container is rebuilt.

window = model.restrict(snapshot=sns[:24], fix={"soc": initial})
window.solve(solver_name="highs")

for start in range(24, len(sns), 24):
    window.advance(snapshot=sns[start : start + 24], fix={"soc": initial})
    window.solve(solver_name="highs")
  • window.matrices exposes the window arrays, with the parent labels of its rows and columns.
  • window.solution and window.dual are Datasets, materialised per container on access, with column, row and objective scaling undone.
  • window.solve builds the native model from the window and updates it in place through ModelDiff.from_windows and each backend's existing _apply_* hooks. A backend without in-place updates rebuilds, and logs a warning; pass disallow_rebuild=True to raise instead.
  • window.advance re-slices the parent; a shape change rebuilds the solver.

Semantics

Limits

  • One selected dimension; several raise.
  • Every carrier of the selected dimension must contain the selected labels.
  • Indicators and quadratic objectives raise; native SOS raises; piecewise works; reformulated SOS still fails on its horizon-wide cardinality row.
  • A window is not a Model: it cannot take new constraints.

Benchmark

Build plus solver assembly, optimisation excluded. 35040 steps, a 1460-step
window, 24 chunks, three variables and three constraints.
hyperfine --warmup 1 --runs 5:

route wall peak RSS
full horizon 514.7 ms ± 7.4 202.6 MiB
chunked rebuild 1236.3 ms ± 11.1 170.2 MiB
restrict + advance 532.3 ms ± 4.6 190.7 MiB

Windowing costs 3.4% over one full build and avoids 24 rebuilds. In-process,
excluding import overhead: 0.170, 0.938 and 0.171 s.

Benchmark script
import sys
import time

import numpy as np
import pandas as pd

import linopy
from linopy.solvers import Solver

N, W = 35040, 1460
route = sys.argv[1] if len(sys.argv) > 1 else "full"


def build(index, prices):
    model = linopy.Model()
    x = model.add_variables(lower=0, coords=[index], name="x")
    y = model.add_variables(lower=0, upper=100, coords=[index], name="y")
    z = model.add_variables(lower=0, coords=[index], name="z")
    model.add_constraints(y - 0.99 * y.shift(time=1) - x + z == 0, name="balance")
    model.add_constraints(x + z <= 50, name="cap")
    model.add_constraints(z - 0.5 * x >= 0, name="mix")
    model.add_objective((prices * x + 0.1 * z).sum())
    return model


def assemble(model):
    Solver.from_name("highs", model=model, io_api="direct")


index = pd.RangeIndex(N, name="time")
prices = pd.Series(np.sin(np.arange(N) / 50), index=index)
fix = {"y": pd.Series(np.zeros(N), index=index)}

start = time.perf_counter()
if route == "full":
    assemble(build(index, prices))
elif route == "chunked":
    for offset in range(0, N, W):
        chunk = index[offset : offset + W]
        assemble(build(chunk, prices.loc[chunk]))
else:
    window = build(index, prices).restrict(time=index[:W], fix=fix)
    for offset in range(W, N, W):
        window.advance(time=index[offset : offset + W], fix=fix)
    assemble(window.as_model_view())

print(f"{route}: {time.perf_counter() - start:.3f}s")

Solver support

Only HiGHS has been exercised by hand. test_window_solve_across_solvers runs
the same recipe on highs, gurobi, cplex, scip, xpress and mosek, skipping
solvers that are absent, file-only, or installed without a license.

solve_window requires io_api="direct". A file-based backend would have to
write the window to disk on every solve, and has no in-place update.

Tests

test_restrict_matrix.py and test_windowing_integration.py, including a solver-parametrized run of the window recipe. linopy/restrict is at 100% statement and branch coverage.

AI assistance

Drafted with Claude Code: the implementation, tests and benchmark. Reviewed by hand before opening.

@olivier-lacroix
olivier-lacroix force-pushed the feat/restrict-matrix-windows branch 2 times, most recently from c48108a to 366d819 Compare October 20, 2026 11:23
@olivier-lacroix
olivier-lacroix marked this pull request as ready for review October 20, 2026 11:26
@codspeed

codspeed Bot commented Oct 20, 2026 •

Copy link
Copy Markdown

Merging this PR will not alter performance

✅ 181 untouched benchmarks
⏩ 181 skipped benchmarks1


Comparing olivier-lacroix:feat/restrict-matrix-windows (9a2af9d) with master (ae80bc8)

Open in CodSpeed

Footnotes

  1. 181 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

@olivier-lacroix
olivier-lacroix force-pushed the feat/restrict-matrix-windows branch 3 times, most recently from 408ba31 to 3da2fd1 Compare October 20, 2026 11:40
@olivier-lacroix

olivier-lacroix commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor Author

I'll resolve typing issues

Edit: done

@FabianHofmann

Copy link
Copy Markdown
Collaborator

@olivier-lacroix thanks for the PR! I now start to have more time to look into other things in linopy. there were some major features which kept me buisy. will take a look at this soon

@olivier-lacroix

Copy link
Copy Markdown
Contributor Author

@FabianHofmann no worries! I can see the big cleanup happening 😄

Olivier Lacroix and others added 2 commits October 29, 2026 20:48
`Model.restrict` returns a `WindowModel` backed by a slice of the parent's
`A`, `b`, `c`, `lb` and `ub` instead of a rebuild of every variable and
constraint. A window is a row slice plus a column range; boundary terms are
folded at their actual coefficient, and containers are materialised only when
their values are read.

- `WindowModel.matrices` exposes the window arrays; `solution`, `dual` and
  `objective` read them back with column, row and objective scaling undone.
- `WindowModel.solve` builds the native model from the window and moves it in
  place through `ModelDiff.from_windows` and the existing per-backend apply
  hooks.
- `advance` re-slices the parent; a shape change rebuilds the solver.
- A window that cuts a coupling without `fix` raises `RestrictionError`, naming
  the constraints.

Supported: one selected dimension, dense and CSR parents, masked rows.
Not supported: indicators, quadratic objectives, SOS, piecewise, several
selected dimensions, subset carriers.

`linopy/restrict` is at 100% statement and branch coverage.

Co-Authored-By: Claude Code <noreply@anthropic.com>
… test

Master removed the autouse semantics fixture with legacy semantics (PyPSA#1041).
The windowing equivalence test still requested it and errored at setup on
every platform.

Co-Authored-By: Claude Code <noreply@anthropic.com>
@olivier-lacroix

olivier-lacroix commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor Author

Rebased onto master at 8736b35 (2026-10-29).

Benchmark rerun on the rebased branch, 5 runs, median:

route wall peak RSS
full horizon 22.1 ms 200.1 MiB
chunked rebuild 209.7 ms 168.5 MiB
restrict + advance 32.4 ms 185.5 MiB

Windowed assembly now costs 47% over one full build (for that small bench). Restrict still costs 6.5x less than 24 rebuilds.

@olivier-lacroix
olivier-lacroix force-pushed the feat/restrict-matrix-windows branch from 9a2af9d to 8048ef2 Compare October 29, 2026 10:01
@codspeed

codspeed Bot commented Oct 29, 2026

Copy link
Copy Markdown

Unable to generate the performance report

Your organization has exceeded its Macro Runners budget. Because this run uses tokenless mode, CodSpeed needs to wait for the GitHub Actions workflow to complete to validate the upload. Since the walltime jobs will never run, the workflow stays pending and we are unable to process this run.

To run these benchmarks again, update your plan or increase your Macro Runners budget.

If you need help, please contact us at support@codspeed.io or on Discord.

…nt, slices and sparsity checks

Windows select container labels via xarray and map them through label_to_pos,
replacing per-container position bookkeeping. fix values align to coords and
honour column scaling; slices and several dimensions work; in-place updates
require an identical sparsity pattern; dropped-row checks use stored signs.
@FabianHofmann

Copy link
Copy Markdown
Collaborator

@olivier-lacroix I would push here if you don't mind?

@FabianHofmann

Copy link
Copy Markdown
Collaborator

still have a major refactor outstanding

@olivier-lacroix

Copy link
Copy Markdown
Contributor Author

Yes of course @FabianHofmann, go for it.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Design: Model.restrict / advance — slice a model along a coordinate and condition on boundary values (rolling horizon)

2 participants