Repository navigation
feat(array): georeferenced arrays, SRID zones, and GeoTIFF COPY - #427
Merged
Merged
Conversation
Add nodedb-raster, which reads GeoTIFF rasters with the deflate and LZW codecs only and refuses JPEG, fax and WebP compression. Decode capacity is bounded before allocation. A fuzz target and its CI job cover the decoder.
Add geometry_from_ewkt with an optional SRID prefix, and orient_rings to normalise exterior and interior ring winding.
Arrays can declare a georeference, which the SQL parser, planner, catalog and cell-geometry functions carry through to region and zone slicing, including the distributed read path. COPY into an array loads GeoTIFF rasters through nodedb-raster. Server-path COPY now requires superuser, reads regular files only and refuses a pre-existing temporary output file. The planner records array catalog versions as unleased descriptors so an ALTER invalidates cached plans, and array retention is tracked in the catalog. The surrogate round-trip integration tests are split into a module directory.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Arrays gain a georeference, polygon zones drawn in a real SRID, and GeoTIFF bulk load through
COPY.What changes
Georeferenced arrays
CREATE ARRAY ... GEOREF (SRID n, X dim, Y dim [, TRANSFORM (gt0, .., gt5)]). WithTRANSFORM, the dims areINT64pixel indices, and the pixel centre is index + 0.5. Without it, the dims hold world coordinates.ALTER ARRAY <name> SET GEOREF (...)andDROP GEOREF. The georef is outsideschema_hash, so data, segments and the Data Plane open check are unchanged.DESCRIBE <array>shows dims, attrs, tile extents and the georef.ARRAY_ELEMENTWISErefuses two arrays with different georefs.Zones in an SRID
ARRAY_AGGandARRAY_SLICEtake an EWKT zone:'SRID=4326;POLYGON((...))'.zone_dimsNULL means the georef axes.ARRAY_SLICEaccepts 2 to 6 arguments (5 is refused). The zone travels to every shard on the distributed read path.ARRAY_CELL_POINT()andARRAY_CELL_FOOTPRINT()return a cell's geometry in the array SRID.GeoTIFF load
COPY <array> FROM '<file>.tif' WITH (FORMAT geotiff [, BANDS (a, SKIP, b)] [, NODATA n] [, TIME '...']).nodedb-rasterontiff0.11.3, with deflate and LZW only. Every decode allocation is bounded before it happens. The first image directory is checked beforetiffreads it.22P04.geotiff_decoderwith a seed corpus, added to the fuzz CI matrix.Behavior changes
COPY(all formats,FROMandTO) requires superuser (42501). It reads regular files only through one handle (58030). Any writer could previously read any file the server user could read.COPY ... FROM STDIN WITH (FORMAT geotiff)is0A000.ALTER ARRAY ... SET (audit_retain_ms = n)refuses negative values. The retention floor is checked again against the committed entry when the ALTER applies.ALTER,DROPor re-CREATEkept running with the old schema.22P04. NewSqlError::FeatureNotSupportedmaps to0A000.Lite
The Lite workspace does not build against this branch until these sites change:
error/from_sql.rs: addSqlError::FeatureNotSupportedto the refused arm.MetaOp::AlterArray.schema_msgpack(physical_visitor/adapter/meta.rs).AlterArrayVisitArgs(adapter/visitor/trait_impl.rs,adapter/visitor/ddl.rs).array_slicetakesArraySliceVisitArgs.ArrayOp::Slicegainszone_msgpack(visitor/array/query.rs,physical_visitor/adapter/array.rs).CreateArrayVisitArgs.georef, andArraySchemaliterals gaingeoref.Measured
tifffileload with row counts and sums equal totifffile's own read.Testing
nodedb-clientfeature sets,cargo deny, every static gate, wasm32 check, and a replay of all 7 fuzz targets.geotiff_decoder: 512,565 inputs, no crash.COPY 49052.ARRAY_AGGover a WGS 84 polygon matches an independent UTM inverse plus point-in-polygon.