Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

writeVector(insert, overwrite) has useful, but undocumented, GeoPackage behavior #1573

Closed
twest820 opened this issue Jul 23, 2024 · 1 comment

Comments

@twest820
Copy link

writeVector()'s documentation for its arguments states

insert      logical. If TRUE, a new layer is inserted into the file, if the format allows it (e.g. GPKG allows that). See vector_layers to remove a layer
overwrite   logical. If TRUE, filename is overwritten

I have a terra 1.7-78 GeoPackage use case with insert for multi-layer file creation

writeVector(layer1data, "bunchOlayers.gpkg", layer = "layer1")
writeVector(layer2data, "bunchOlayers.gpkg", layer = "layer2", insert = TRUE)
[...]

For updating individual layers overwrite is needed

# updates layer1, does not change rest of bunchOlayers.gpkg
writeVector(layer1data, "bunchOlayers.gpkg", layer = "layer1", insert = TRUE, overwrite = TRUE)
# updates layer2, does not change rest of bunchOlayers.gpkg
writeVector(layer2data, "bunchOlayers.gpkg", layer = "layer2", insert = TRUE, overwrite = TRUE)
[...]

where overwrite behavior is consistent with GeoPackage layer updating in other GDAL based tools such as QGIS. But, as each individual layer update doesn't obliterate all the other layers in the package, is reasonably interpreted as inconsistent with terra's documentation. IMO this is a feature as it's pretty handy to be able to update one layer without having to buffer a GeoPackage's entire contents to memory and then insert all the other layers back as well. However, usual practice in defensive programming is to avoid undocumented behavior as it's not part of the API contract and thus subject to change.

An apparent corollary to this is specifying just overwrite = TRUE doesn't behave as documented.

writeVector(layer1data, "bunchOlayers.gpkg", layer = "layer1", overwrite = TRUE)
Error: [writeVector] Creation of output dataset failed
In addition: Warning message:
A file system object called 'bunchOlayers.gpkg' already exists. (GDAL error 1) 

I'm thus wondering if terra's documentation should be amended to something like

insert      logical. If TRUE, a new layer is inserted or an existing layer overwritten if the file format allows it (e.g. GPKG allows these), depending on the value of overwrite. See vector_layers to remove a layer
overwrite   logical. If TRUE and insert is FALSE, filename will be overwritten if the file format and layer structure permits. If TRUE and insert is TRUE, only the target layer is overwritten when the format allows (e.g. GPKG).

with then perhaps some remarks about behavior with GeoPackages and other common formats. This would make the ability to update individual layers in GeoPackages a (formally) supported behavior.

@rhijmans
Copy link
Member

Thank you very much for the thorough analysis and clear suggestions, that is very helpful.

netbsd-srcmastr pushed a commit to NetBSD/pkgsrc that referenced this issue Jan 5, 2025
# version 1.8-5

## bug fixes

- `Spatsample(method='stratified', ext=e)` returned the wrong sampling
  coordinates [#1628](rspatial/terra#1628)
  by Barnabas Harris

- `spatSample(method='stratified')` could fail with small sample sizes
  [#1503](rspatial/terra#1503) by karluf

- transparency (alpha) did not work with RGB
  plotting. [#1642](rspatial/terra#1642) by
  Timothée Giraud

- rasterization failed on very large rasters
  [#1636](rspatial/terra#1636) by Mary
  Fisher, [#1463](rspatial/terra#1463) by
  Nic Spono and [#1281](rspatial/terra#1281)
  by Sebastian Dunnett

- `tmpFiles` only looked in the default temp files folder
  [#1630](rspatial/terra#1630) by
  smckenzie1986

- `where.min` did not work well if there were negative values
  [#1634](rspatial/terra#1634) by Michael
  Sumner

- `plet<SpatRaster>` now works for RGB rasters and rasters with a
  color table [#1596](rspatial/terra#1596)
  by Agustin Lobo

- `vect<MULTIPOINT WKT>` did not work properly
  [#1376](rspatial/terra#1376) by
  silasprincipe

- `compareGeom<SpatVector>` did not work
  [#1654](rspatial/terra#1654) by Jason
  Flower

- `buffer<SpatVector>` is now more accurate buffers for lonlat
  polygons [#1616](rspatial/terra#1616) by
  Roberto Amaral-Santos

- `terra:interpNear` used square windows, not circles, beyond 100
  points [#1509](rspatial/terra#1509) by
  Jean-Luc Dupouey

- `vect` read INT64 fields as integers, sometimes leading to
  overflows. [#1666](rspatial/terra#1666) by
  bengannon-fc

- `plot` showed a legend title even if none was requestd if title
  parameters were specified
  . [#1664](rspatial/terra#1664) by Márcia
  Barbosa



## enhancements

n- improved documentation of `writeVector` overwrite when using
  layers. [#1573](rspatial/terra#1573) by
  Todd West

- improved treatment of (supposedly) flipped rasters by Timothée
  Giraud [#1627](rspatial/terra#1627) and
  fchianucci [#1646](rspatial/terra#1646)

- added `map.pal("random")`
  [#1631](rspatial/terra#1631) by Agustin
  Lobo

- expressions can now be used in legend titles
  [#1626](rspatial/terra#1626) by Noah
  Goodkind

- `app` and `tapp` now emit a warning when factors are coerced to
  numeric [#1566](rspatial/terra#1566) by
  shuysman

- `plet<SpatRaster>` now has argument "stretch" for RGB rasters
  [#1596](rspatial/terra#1596) by Agustin

- `%%` and `%/%` now behave the same for SpatRaster as for (base R)
  numbers [#1661](rspatial/terra#1661) by
  Klaus Huebert

## new

- `patches` with option `valus=TRUE` can now distinguish regions based
  on their cell values (instead of only NA vs not-NA)
  [#495](rspatial/terra#495) by Jakub
  Nowosad and [#1632](rspatial/terra#1632)
  by Agustin Lobo

- `rowSums`, `rowMeans`, `colSums` and `colMeans` for SpatRaster

- `metags` for SpatRasterDataset
  [#1624](rspatial/terra#1624) by Andrea
  Manica

- `metags` for layers (bands) of SpatRaster are now saved to and read
  from GTiff files
  [#1071](rspatial/terra#1071) by Mike
  Koontz

- `global` has new effcient functions "anyNA" and "anynotNA"
  [#1540](rspatial/terra#1540) by Kevin J
  Wolz

- `wrap`, `saveRDS` and `serialize` for
  SpatExtent. [#1430](rspatial/terra#1430)
  by BastienFR

- `vect<SpatGraticule>` method suggested in relation to [tidyterra
  #155](dieghernan/tidyterra#155) by Diego
  Hernangómez

- `toMemory<SpatRaster>` and `<SpatRasterDataset>` methods
  [#1660](rspatial/terra#1660) by Derek Friend
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants